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Overview 


Developmental summer bridge programs are a popular strategy for increasing college 
readiness among recent high school graduates. Aimed at providing an alternative to 
traditional developmental education, these programs provide accelerated and focused 
learning opportunities in order to help students acquire the knowledge and skills needed for 
college success. 

The current study uses an experimental design to evaluate the outcomes of eight 
developmental summer bridge programs offered in Texas during the summer of 2009. At 
each college, students who consented to participate in the study were randomly assigned to 
either a program group that was eligible to participate in a developmental summer bridge 
program or a control group that was eligible to use any other services that the college 
provided. Based on a program model developed by the Texas Higher Education Coordinating 
Board, the developmental summer bridge programs in this study included four common 
features: accelerated instruction in developmental math, reading, and/or writing; academic 
support; a “college knowledge” component; and the opportunity to earn a $400 stipend. 

After two years of follow-up, these are the main findings of this study: 

• The programs had no effect on the average number of credits attempted 
or earned. Program group and control group students attempted the 
same number of credits (30.3). Students in the program group earned an 
average of 19.4 credits, and students in the control group earned an 
average of 19.9 credits; the difference in their outcomes is not 
statistically significant. 

• The programs had an impact on first college-level course completion in 
math and writing that was evident in the year and a half following the 
program but no impact on first college-level course completion in 
reading during this same period. On average, students in the program 
group passed their first college-level math and writing courses at higher 
rates than students in the control group during this period. By the end 
of the two-year follow-up period, however, the differences between the 
two groups are no longer statistically significant. 

• There is no evidence that the programs impacted persistence. During the 
two-year follow-up period, students in the program group enrolled in an 
average of 3.3 semesters, and students in the control group enrolled in an 
average of 3.4 semesters, a difference that is not statistically significant. 
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Preface 


Nationally, a significant portion of college students are deemed unprepared for 
college-level work in at least one subject area and are required or encouraged to enroll in 
developmental programs. A subset of these students — those with especially weak 
academic skills — must often complete two or more semesters of remedial coursework 
before attempting college-level courses in the relevant subject areas. But there is growing 
evidence that lengthy sequences of remedial coursework may not be beneficial for students. 
Few students who embark on a multi-course sequence complete those courses, and even 
fewer earn a postsecondary credential. 

As a result, many colleges are pursuing innovative alternative approaches to 
developmental education that aim to accelerate students’ progress in gaining important 
academic competencies. Summer bridge programs are one such approach. They offer 
underprepared students a chance to advance toward college-level coursework during the 
summer before they begin college. In the past several years, summer bridge programs have 
grown increasingly popular as a strategy for providing students with the knowledge and 
skills required for college success. Until recently, however, there has been little rigorous 
empirical research on their effectiveness. 

NCPR’s study of eight developmental summer bridge programs in Texas is the first 
on this subject to employ an experimental design. Using a randomized controlled trial, this 
study examined the impact of program participation over the course of two years — long 
enough to make some meaningful observations about how the programs influenced student 
outcomes in college. It is clear from the study’s results that the program model was more 
successful in achieving short-term gains than it was in realizing long-term advantages for 
students. Program group students were more likely than control group students to pass their 
introductory college-level math and writing courses during the first few semesters in college 
after participation, but these higher rates of first college-level course completion began to 
diminish in the final semesters of the two-year follow-up period. Moreover, program group 
students were no more likely than control group students to persist in college or to earn 
more credits. 

In exploring the meaning of these findings, the authors consider their implications 
for short-term programmatic interventions in general. If the developmental summer bridge 
programs in this study — which were well-established and well-implemented — produced 
only modestly positive effects that began to fade after a few semesters, can we reasonably 
expect any type of short-term intervention to have sizeable long-term effects? And if not, 
how can students best be offered the support they need as they work toward their long-term 
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academic goals? These considerations, along with the study’s findings, will be of interest to 
policymakers and college leaders looking to reduce the time students spend in 
developmental education and increase the number of students who finish college. 


Thomas Bailey 
Director, NCPR 
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Executive Summary 


Across the country, a growing number of recent high school graduates are 
participating in summer bridge programs. These programs provide accelerated and 
focused learning opportunities in order to help students acquire the knowledge and skills 
needed for college success. The state of Texas has given particular attention to summer 
programs as a way to increase students’ college readiness. During the past several years, 
the Texas Higher Education Coordinating Board (THECB) has provided support to 
colleges establishing developmental summer bridge programs offering intensive 
remedial instruction in math, reading, and/or writing, along with an introduction to 
college. In contrast with traditional developmental education course sequences, which 
may span several semesters, the summer bridge programs were designed to help 
underprepared students build competencies over the course of several weeks before 
entering college. 

While THECB funding for summer bridge programs has diminished, this type of 
program model remains popular in Texas and across the country. Nevertheless, little 
rigorous empirical research has been conducted on the effectiveness of summer bridge 
programs (Ackermann, 1990; Garcia, 1991; Myers & Drevlow, 1982; Santa Rita & Bacote, 
1997). To address this gap in the research, in 2009 the National Center for Postsecondary 
Research (NCPR) 1 launched an evaluation of summer bridge programs at eight sites in 
Texas to assess whether they reduce the need for developmental coursework upon fall 
matriculation and improve student outcomes in college. 


The Developmental Summer Bridge Programs 

The developmental summer bridge programs in this study were offered in the 
summer of 2009, primarily to recent high school graduates, at eight institutions of higher 
education — two open-admissions four-year institutions and six community colleges. 
Students attended the developmental summer bridge programs for three to six hours daily 
for four to five weeks and received instruction in at least one area of academic need — 
math, reading, or writing — and guidance in the “college knowledge” needed to navigate 


'NCPR is a partnership funded by the Institute of Education Sciences of the U.S. Department of 
Education from 2006 to 2012. NCPR includes the Community College Research Center at Columbia 
University’s Teachers College, MDRC, the Curry School of Education at the University of Virginia, and 
faculty at Harvard University. NCPR conducts studies that measure the effectiveness of programs 
designed to help students make the transition to college and master the basic skills needed to advance to a 
degree. Houston Endowment also contributed funds to support this research. 
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new academic terrain. All of the developmental summer bridge programs included four 
common features: accelerated instruction in math, reading, and/or writing; academic 
support; a college knowledge component; and the opportunity to earn a $400 stipend. 


The Research 

The evaluation employed an experimental design to measure the effects of the 
programs on college enrollment and success. At each college, students who consented to 
participate in the study were randomly assigned to either a program group that was eligible 
to participate in a developmental summer bridge program or a control group that was 
eligible to use any services that the college provided other than the summer bridge 
programs. Random assignment creates two groups that are similar on all characteristics, 
including those that can be measured, such as age or academic attainment, and those that 
are more difficult to measure, such as motivation. This ensures that any differences in 
observed outcomes — called impacts — can be attributed to participation in the 
developmental summer bridge programs. 

Eligible students who applied for admission into a developmental summer bridge 
program and agreed to participate in the study were included in the research sample. After 
consenting to participate and completing a baseline intake form, these students were 
randomly assigned to either the program group or the control group. About 60 percent of 
the students were assigned to the program group and given the opportunity to take one of 
the available slots in the summer bridge program (793 students), while about 40 percent 
were assigned to the control group and were able to participate in other college services but 
were not admitted to the program itself (525 students). Students in both groups consented to 
have their outcomes tracked for two full academic years. 

NCPR collected and analyzed academic outcome data through the spring semester 
of 2011 for both program and control group students. This report presents the impact 
findings of the study, revealing whether the opportunity to participate in a summer bridge 
program influenced academic outcomes during the two years following participation. The 
primary outcomes tracked in this study were persistence, accumulation of credits, and 
progression through the developmental sequence and into students’ first college-level math, 
reading, and writing courses. 


2 

"For more information on the implementation of the programs, see Wathington et al. (2011). 
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Main Findings 

After two years of follow-up, these are the main findings of this study: 

• The programs had no effect on the average number of credits attempted 
or earned. Program group and control group students attempted the 
same number of credits (30.3). Students in the program group earned an 
average of 19.4 credits, and students in the control group earned an 
average of 19.9 credits; the difference in their outcomes is not 
statistically significant. 

• The programs had an impact on first college-level course completion in 
math and writing that was evident in the year and a half following the 
program but no impact on first college-level course completion in 
reading during this same period. On average, students in the program 
group passed their first college-level math and writing courses at higher 
rates than students in the control group during this period. By the end 
of the two-year follow-up period, however, the differences between the 
two groups are no longer statistically significant. 

• There is no evidence that the programs impacted persistence. During 
the two-year follow-up period, students in the program group enrolled 
in an average of 3.3 semesters, and students in the control group 
enrolled in an average of 3.4 semesters, a difference that is not 
statistically significant. 


Program Costs 

NCPR performed an analysis of the cost of the developmental summer bridge 
program. The sites varied in terms of program duration, intensity, and enrollment, and total 
costs to run the program during the summer of 2009 ranged from $62,633 to $296,033. 
Across the eight sites, per student costs ranged from $835 to $2,349. The average cost per 
student across all eight sites was $1,319 (with a standard deviation of $502). 3 

We also calculated the college-level credit accumulation that the developmental 
summer bridge programs would have had to produce in order to be cost effective on this 
outcome measure. Specifically, we considered how many additional college credits a 


3 Some costs may be interpreted as “start-up'’ costs, which are unlikely to be needed if the programs 
are run in subsequent years. If these costs are amortized over three years, then the average cost of the 
programs is reduced. In addition, this figure includes the student stipend of up to $400 per participant. 
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developmental summer bridge program student would need to earn to justify the cost of the 
program. In order to do this, we assigned a monetary cost of $338 to college credits earned, 
based on the typical cost of providing these credits in Texas. 4 The program group would 
have had to earn an additional 3.8 college-level credits on average for the program to justify 
its costs or “break even.” 


Implications 

The findings in this report suggest that the developmental summer bridge programs 
contributed to positive outcomes in college-level course completion in math and writing 
that were evident during the first year and a half after program completion. However, the 
programs did not lead to increases in persistence or overall credit completion, raising the 
question of whether our theory of change and the changes in measured outcomes that we 
hypothesized were reasonable were too ambitious. It may be that we should not expect to 
find long-term impacts on credit accumulation and persistence from a short, intensive 
summer program. First-year developmental education students may need further support for 
greater impacts to be achieved. 

In addition, our research suggests that accelerating students’ completion of 
introductory college-level courses in math or English may not lead to the accumulation of 
more college credits overall. If the ultimate goal is college credential attainment, and credit 
accumulation indicates progress toward attaining a credential, improving academic 
preparedness through developmental summer bridge programs or other similar programs 
may not adequately promote attainment of this goal. Policymakers and practitioners 
concerned with college completion may want to consider approaches that go further in 
assisting students in ongoing credit accumulation and credential attainment. 

Finally, our break-even cost analysis suggests that students in the developmental 
summer bridge programs would need to have earned an average of almost four additional 
college credits to justify the cost of the program (courses are typically worth three credits). 
Given that no impact on credit accumulation was found, college practitioners and 
policymakers may reasonably view the programs as expensive. Educators may want to 
consider if there are ways to reduce costs by embedding support programs such as these 
into the regular high school or college schedule. 


4 This is the average of the expenditure per credit across seven of the eight colleges based on 
Integrated Postsecondary Education Data System (IPEDS) data of expenditures per FTE (2008 data 
uprated to 2011 dollars). Expenditures per FTE are adjusted to capture expenditures per credit attempted. 
One college did not have available IPEDS data. 
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Concluding Thoughts 

Similar to other innovative developmental education programs that have been 
rigorously evaluated, 5 the developmental summer bridge programs studied here were 
found to have modest positive impacts in the short term. What is clear from this study and 
other developmental education research is that simple, short-term interventions yielding 
strong, long-term effects are difficult to find. With this in mind, we offer two suggestions 
for advancing the work of supporting underprepared students: (1) introducing new 
partnerships between high schools and colleges that reduce the need for remediation in 
college and (2) providing more support and transitional experiences to help students reach 
and sustain attainment goals. Because educational attainment is the result of a long 
process influenced by many factors, providing supports to students that span their years in 
high school and college may help them to develop the skills and knowledge required for 
postsecondary success. 


5 See, for example, findings from NCPR’s Learning Communities Demonstration (Visher, Weiss, 
Weissman, Rudd, & Wathington, 2012). 
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Chapter 1 


Introduction 


In 1972, Alice Cooper declared that “school’s out for summer” (Cooper, 1972, track 
1). For many young students, these lyrics still capture the sense of freedom that 
accompanies their yearly reprieve from academic work. For a great number of recent high 
school graduates, however, the summer before college matriculation is no time for 
relaxation, nor is it an opportunity to work full time to save for future endeavors. Across the 
country, a growing number of recent high school graduates are participating in summer 
bridge programs. These programs provide accelerated and focused learning opportunities in 
order to help students acquire the knowledge and skills needed for college success. 

Texas has provided support for programs to increase students’ college readiness as 
part of a statewide initiative called Closing the Gaps by 2015, which is designed to increase 
rates of college participation and success. In 2007, the Texas Higher Education 
Coordinating Board (THECB) provided funding to 22 colleges to establish developmental 
summer bridge programs. These intensive summer programs offered eligible students 
remedial instruction in math, reading, and/or writing along with an introduction to college. 
In contrast with traditional developmental education course sequences, which may span 
several semesters, the summer bridge programs were designed to help underprepared 
students build competencies over the course of several weeks before entering college. 

In Texas and across the country, summer bridge programs have become a popular 
strategy for increasing college readiness among recent high school graduates. In fact, a 
number of two- and four-year colleges in Texas have developed their own summer bridge 
programs independent of the guidelines and funds provided by the THECB . However, little 
rigorous empirical research on their effectiveness has been conducted in Texas or elsewhere 
(Ackermann, 1990; Garcia, 1991; Myers & Drevlow, 1982; Santa Rita & Bacote, 1997). 

In 2009, the National Center for Postsecondary Research (NCPR) 1 launched an 
evaluation of summer bridge programs at eight sites in Texas to assess whether they 
reduce students’ need for developmental coursework upon fall matriculation and improve 
student outcomes in college. The evaluation used an experimental design to measure the 


'NCPR is a partnership funded by the Institute of Education Sciences of the U.S. Department of 
Education from 2006 to 2012. NCPR includes the Community College Research Center at Columbia 
University’s Teachers College, MDRC, the Curry School of Education at the University of Virginia, and 
faculty at Harvard University. NCPR conducts studies that measure the effectiveness of programs designed 
to help students make the transition to college and master the basic skills needed to advance to a degree. 
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effects of the programs on college enrollment and success. At each college, students who 
consented to participate in the study were randomly assigned to either a program group 
that was eligible to participate in a developmental summer bridge program or a control 
group that was eligible to use any services that the college provided other than the 
summer bridge programs, including regular courses, orientations to college, counseling, 
etc.” Random assignment creates two groups that are similar on all characteristics, 
including those that can be measured, such as age or academic attainment, and those that 
are more difficult to measure, such as motivation. This ensures that any differences in 
observed outcomes — called impacts — can be attributed to participation in the 
developmental summer bridge programs. 

NCPR collected academic outcome data through the spring semester of 2011 for 
both program and control group students. This report presents the impact findings of the 
study, revealing whether the opportunity to participate in a summer bridge program 
influenced academic outcomes during the two years following participation. 


The Potential of Summer Bridge Programs 

While none of the prior research on summer bridge programs employed rigorous 
experimental designs, some evidence suggests that underprepared students who participate 
in these programs show improvement in their academic performance (Bengis, 1991; 
Kallison & Stader, 2012; Strayhorn, 2011). Proponents of summer bridge programs suggest 
that there are several ways in which these programs may be more effective than traditional 
models of developmental education in helping students with below-average skill sets to 
build competencies and succeed in college. 

First, the accelerated instruction provided by summer bridge programs may allow 
students at-risk of not completing college to advance through developmental education 
more quickly, ideally enabling recent high school graduates to enroll in college-level 
courses in the fall of their first year of college (Edgecombe, 2011). This may have a 
beneficial effect on subsequent academic outcomes; empirical evidence suggests that the 
length of time students spend in developmental education courses is negatively related to 
degree completion (Adelman, 1998; Attewell, Lavin, Domina, & Levey, 2006). One 
hypothesis that would explain this relationship is that developmental courses may 
discourage students and lead them to drop out. If this is the case, enrolling students in 


2 

Enrollment data from the eight participating colleges show that 16 percent of students in the control 
group enrolled in a course at the college where they applied for admission to a summer bridge program 
during the summer of 2009. 
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college-level courses as soon as possible after high school may improve students’ 
probability of degree attainment. 

Another reason that developmental summer bridge programs may be more effective 
than traditional developmental education sequences is that the former offer an array of 
support services to ease the transition, both academically and socially, from high school to 
college. Research suggests that at-risk students benefit from non-academic support 
programs that help them create social relationships, clarify their aspirations and academic 
commitments, and develop college know-how (Karp, 2011). Summer bridge programs may 
help participants develop non-academic skills in several ways. Introducing at-risk students 
to the college environment and actively teaching them how to navigate and take advantage 
of services may improve their likelihood of persistence (Deil-Amen & Rosenbaum, 2003; 
Rosenbaum, Deil-Amen, & Person, 2006; Tinto, 1993). In addition, through summer bridge 
programs, students have the opportunity to form relationships with mentors, peers, and 
faculty that may strengthen their academic and social support networks as well as their ties 
to the institution, which may in turn promote student persistence. 

Finally, by offering a stipend to students for their participation, summer bridge 
programs may encourage students to take on an academic endeavor rather than a summer 
job. Stipends may also help to defray the costs of college. 


Key Facts and Findings 

NCPR began its work in Texas by establishing a cooperative agreement with the 
THECB. A number of developmental summer bridge programs had received THECB 
funding and technical assistance for two years prior to the beginning of the research, 
resulting in a pool of colleges with well-implemented, fully realized programs. NCPR and 
the THECB agreed to cooperate on the logistics of this study and to share data that could be 
used in assessing the implementation and impact of these programs. Houston Endowment 
also was interested in increasing participation and success rates in college and became a 
supporting partner in this study. 

Following an intensive period of investigation into the kinds of programs offered 
around the state, eight colleges offering well-established developmental summer bridge 
programs were invited to participate in the research. NCPR worked closely with these 
colleges to refine program designs and recruit students; many of the colleges aimed to 
enroll more students than they did in previous years. The programs targeted students who 
had placed into developmental education based on their high school exit exam or college 
placement test scores. Interested students were randomly assigned to participate in the 
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available spaces in the eight programs. Sixty percent of students were assigned to the 
program group and 40 percent to the control group (See Table 1.1). 

Prior to random assignment, all students gave consent for NCPR to obtain their 
enrollment and transcript data. Access to these data permitted researchers to assess whether 
participation in a summer bridge program affected key student outcomes, such as need for 
developmental education, credit accumulation, and persistence. In addition, NCPR 
researchers made site visits to participating colleges to learn how the summer bridge 
programs were designed and implemented. A prior report on this research provides 
information on program implementation and initial impact findings (Wathington et al., 
2011). The current report provides a summary of student outcomes after following the 
students for two full academic years. 


Texas Developmental Summer Bridge Programs 
Table 1.1 


Colleges Participating in the Study 


Institution 

Location 

Program 

Students 

(60%) 

Control 

Students 

(40%) 

Total Students 
Randomly 
Assigned 

El Paso Community College 

El Paso 

165 

108 

273 

Lone Star College-CyFair 

Houston 

74 

48 

122 

Lone Star College-Kingwood 

Houston 

51 

35 

86 

South Texas College 

McAllen 

83 

54 

137 

Texas A&M International University 

Laredo 

126 

85 

211 

Palo Alto College 

San Antonio 

52 

35 

87 

San Antonio College 

San Antonio 

89 

58 

147 

St. Philip’s College 

San Antonio 

153 

102 

255 

Total 


799 

533 

1,318 


NOTE: More students were recruited ( n = 1,332) than participated in the study (n = 1,318). 


4 



After two years of follow-up, these are the main findings of this study: 

• The programs had no effect on the average number of credits attempted 
or earned. Program group and control group students attempted the 
same number of credits (30.3). Students in the program group earned an 
average of 19.4 credits, and students in the control group earned an 
average of 19.9 credits; the difference in their outcomes is not 
statistically significant. 

• The programs had an impact on first college-level course completion in 
math and writing that was evident in the year and a half following the 
program but no impact on first college-level course completion in 
reading during this same period. On average, students in the program 
group passed their first college-level math and writing courses at higher 
rates than students in the control group during this period. By the end 
of the two-year follow-up period, however, the differences between the 
two groups are no longer statistically significant. 

• There is no evidence that the programs impacted persistence. During 
the two-year follow-up period, students in the program group enrolled 
in an average of 3.3 semesters, and students in the control group 
enrolled in an average of 3.4 semesters, a difference that is not 
statistically significant. 


Organization of This Report 

The following chapters describe the study and its findings. Chapter 2 provides 
information on the design of the developmental summer bridge programs. Chapter 3 details 
the study’s experimental methodology, data collection procedures, and analyses. The 
impact findings on primary and secondary outcomes, as well as subgroup analyses, are 
provided in Chapter 4. Chapter 5 reviews the developmental summer bridge program’s 
costs and its potential for cost-effectiveness. Finally, Chapter 6 discusses the implications of 
these findings for policy, practice, and research. 
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Chapter 2 


The Developmental Summer Bridge Program Model 

The developmental summer bridge programs in this study were offered in the 
summer of 2009, primarily to recent high school graduates 1 2 at eight institutions of higher 
education — two open-admissions four-year institutions and six community colleges. 
Students attended the programs for three to six hours daily for four to five weeks and 
received instruction in at least one area of academic need — math, reading, or writing — 
and guidance in the “college knowledge” needed to navigate new academic terrain. The 
programs were free for students at five of the eight sites.” All of the programs included four 
common features: accelerated instruction in math, reading, and/or writing; academic 
support; a college knowledge component; and the opportunity to earn a $400 stipend. 3 

Table 1.1 shows the eight participating institutions and the number of students they 
recruited. Of the eight developmental summer bridge programs included in the study, four 
were course-based , , while the other four were, freestanding 4 (see Table 2.1). Course-based 
programs were essentially standard developmental courses, modified or condensed to fit the 
summer timeframe. Freestanding programs provided students the opportunity to advance 
multiple levels in the developmental education sequence, but they did not require students 
to enroll in a developmental educational course over the summer. Students in freestanding 
programs did not receive developmental education credit for their participation. Students in 
course -based programs were enrolled at the college and did receive developmental 
education credit upon successful course completion/ Freestanding programs had more 
opportunities to make changes in content or instructional methods than did course-based 
programs, which followed established course outlines. 


1 All colleges reached out to students who were likely to enroll (or already registered) in the fall. 
Some of the colleges made significant efforts to recruit students who were undecided about college 
attendance in hopes that the summer bridge program could provide an extra impetus for them to attend 
college in the fall. 

2 The two Lone Star College sites (CyFair and Kingwood) charged students $150 to participate in the 
developmental summer bridge program. The funds were deducted from the student stipend. San Antonio 

College charged tuition but helped students to obtain financial aid to offset the cost of participation. 

3 

For more information on the implementation of the programs, see Wathington et al. (2011). 

4 The program at St. Philip’s College is classified as a freestanding program because students did not 
receive developmental education credit for participation. However, students were required to enroll in St. 
Philip's over the summer in order to participate in the program. In NCPR's first report on the 
developmental summer bridge programs, St. Philip’s was characterized as a course-based program. 

^Developmental education credits are awarded for financial aid eligibility purposes but cannot be 
applied toward a degree. 
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Texas Developmental Summer Bridge Programs 
Table 2.1 


Characteristics of Developmental Summer Bridge Programs 


Institution 

Hours of 
Instruction 

Weeks of 
Instruction 

Course- 

Based? 

Subjects 

Offered 

Levels Below 
College Level 

Student 
Ability Levels 

El Paso Community 
College 

100 

5 

No 

Math, Reading, 
and Writing 

All 

Mixed 

Lone Star College- 
CyFair 

67 

4 

Yes 

Math, Reading, 
and Writing 

1 

Not Mixed 

Lone Star College- 
Kingwood 

52 (Writing) 
64 (Math) 

4 

Yes 

Math and 
Writing 

1 

Not Mixed 

South Texas College 

80-100 

4 

No 

Math 

All 

Mixed 

Texas A&M 
International University 

100 

5 

No 

Math 

All 

Mixed 

Palo Alto College 

60-76 

4 

Yes 

Math 

2 and 3 
(lowest) 

Not Mixed 

San Antonio College 

Approx. 

97 

5 

Yes 

Math, Reading, 
and Writing 

All 

Not Mixed 

St. Philip’s College 

Approx. 

35-95 

4 

No 

Math, Reading, 
and Writing 

All 

Mixed 


As with a standard developmental course, students enrolled in course -based 
programs were graded based on performance; students who earned a C or better earned 
developmental credit and were allowed to advance to the next course in the sequence. 
Students in the freestanding programs were not given traditional grades. Rather, they 
worked toward retaking and passing the institution’s placement test (COMPASS, 
ACCUPLACER, etc.) at the end of the program. All programs were designed to help 
developmental students advance at least one level of proficiency. 

Table 2.1 shows the characteristics of each developmental summer bridge program 
included in the study. Though the eight programs shared many common elements, each had 
unique features to accommodate the institution’s student population or, in some cases, to fit 



with the academic calendar or institutional culture. This section describes each of the four 
core program features in detail. Descriptions of the programs at the individual colleges are 
included in Appendix B . 


Developmental Summer Bridge Program Features 

Accelerated Instruction in Math, Reading, and/or Writing 

All sites offered developmental education instruction in an accelerated format, with 
some degree of contextualized and active learning. Formats, schedules, and content differed 
from site to site and were influenced by subject areas taught and by program goals. South 
Texas College, Texas A&M International University (TAMIU), and Palo Alto College 
offered only math; Lone Star College-Kingwood offered math and writing; and St. Philip’s 
College, El Paso Community College (EPCC), Lone Star College-CyFair, and San Antonio 
College offered a combination of math, reading, and writing. A few programs taught 
students of different developmental levels together in a freestanding program. 

The length and intensity of the developmental summer bridge programs varied, as 
shown in Table 2.1. While all programs ran for four to five weeks, the hours spent in class 
differed. The minimum time required to complete a program was about 35 hours for a student 
taking only one subject at St. Philip’s College, though a large majority of this college’s 
students took two courses for over 50 hours of program time. Students enrolled at EPCC or 
TAMIU spent approximately 100 hours on campus. These hours represent required time at 
the college and do not include hours spent on homework or group project assignments. 

Academic Support 

Academic support was an important part of the developmental summer bridge 
programs, not only because it supported the instructional component but also because it 
helped to integrate students into the life of the program and the college. This support 
generally took the form of tutoring, mentoring, or access to learning labs and computer- 
based programs. The math programs generally utilized some form of mandatory computer- 
aided instruction and independent practice in a lab setting, as did some of the reading and 
writing programs. If students had Internet access outside of school, they could log on 
remotely, but this was not required by any of the programs. 

All of the programs except for San Antonio College and St. Philip’s College 
provided tutors as part of the summer bridge program. Programs used varied approaches to 
tutoring, such as assigning a tutor to each class for a few hours per week, placing a tutor in 
every classroom for the entire period, or having three tutors roaming among five classes. 
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While San Antonio College did not provide tutors, math students were encouraged to spend 
ten hours per week in the math lab, where tutoring was available. 

College Knowledge Component 

College knowledge instruction is defined as sharing information about college 
contexts (e.g., through tours or introductions to student services), college expectations (e.g., 
those regarding study skills, classroom norms, and time management), college planning 
(e.g., course-taking, plans for transfer), aligning educational goals with career plans, and 
paying for college. All programs in the study provided some explicit means to help students 
gain college knowledge, but the specific content and delivery method varied. 

College knowledge can be divided into social and academic components. In the 
developmental summer bridge programs, academic knowledge was stressed more than 
social knowledge. The programs covered a variety of academic topics, including study and 
test-taking strategies, time management, career assessment, learning styles, tours of the 
campus, introduction to college resources, financial aid, and course or degree plans. The 
information needed to make the social transition to the college was covered to some degree 
in all summer bridge programs, with some instruction provided in social college knowledge 
topics such as personal financial responsibility, motivation, behavioral expectations, and 
stress management. 

Content was formally delivered in two main ways. Three programs (Palo Alto 
College, San Antonio College, and St. Philip’s College) offered a modified version of a 
student development course (sometimes referred to as Student Success or College 101) of 
the type typically offered to new college students. These courses were often taught by 
college advisors and had a designated time slot in the day’s schedule. The other five 
programs provided presentations focused on different college knowledge topics, such as 
financial aid and career assessments. These colleges generally offered four or five one-hour 
presentations, once per week. Presentations were not generally integrated into the 
curriculum and were taught by either advisors or mentors. 

In addition, some programs used mentors to help students better understand the 
transition to college. Structured mentoring was offered at EPCC, TAMIU, Lone Star 
College-CyFair, and Lone Star College-Kingwood. EPCC and TAMIU had strong 
mentoring programs in which students met with mentors at specific times to cover a series 
of topics identified in advance. This structure appears to have maximized the effectiveness 
of the time students spent in mentoring sessions. The two Lone Star Colleges assigned their 
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College Connection 6 advisors as mentors because these individuals had extensive 
experience in high schools. The Lone Star Colleges did not have a structured training 
program or predetermined topics for the mentors to cover with students. 

While most of the programs offered some assistance with financial aid, assistance in 
the preparation of financial aid forms was explicitly provided at several colleges. The 
Alamo Colleges (San Antonio, Palo Alto, and St. Philip’s) helped students to complete the 
Free Application for Federal Student Aid (FAFSA) as they were enrolling in the program. 
Completing this application enabled students to receive financial aid to cover program costs 
and to obtain the funds needed to further pursue their education. 

Stipend 

All of the features described above were a part of the intensive summer experience 
model framed by the THECB. The addition of the opportunity to earn a $400 stipend was 
initiated and funded by NCPR. The stipend incentive was built into the program design to 
boost recruitment and increase the likelihood that students would complete the program 
once enrolled. Students typically received $150 at the beginning of the program and $250 
once they successfully completed the program. Each college independently defined 
successful completion. More information about the stipend can be found in Chapter 4. 


Theory of Change 

Before we began studying developmental summer bridge programs, we 
hypothesized that these programs would have a positive impact on students’ academic 
outcomes through several mechanisms. These were the basis upon which the outcome 
measures used in this research were selected. 

First, we presumed that students would be academically and socially better prepared 
for college after attending the summer bridge program. We hypothesized that this 
preparation — combined with better knowledge of and comfort with college enrollment 
procedures, campus facilities, and other aspects of college life — would lead to higher rates 
of enrollment in the fall and persistence in subsequent semesters. We also believed that it 
would lead to higher rates of credit accumulation. 

We also hypothesized that students’ early and accelerated exposure to developmental 
coursework would lead them to test out of developmental course requirements or begin 


6 College Connection advisors are used throughout the state to assist high school students in the 
transition to college. They are typically employees of the colleges and based in the high schools. 
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college further along the developmental course sequence. This would allow them to earn and 
accumulate college-level credits sooner than they would have been able to without the 
program. Further, we expected that students would be more likely to utilize on-campus 
services, which would influence their pass rates and thus their credit accumulation. 

Finally, we conjectured that students would be more likely to apply for and 
receive financial aid, specifically grants and subsidized loans (which are seen as “better” 
forms of aid than unsubsidized and/or private loans). This could support persistence and 
credit accumulation by allowing students to afford to take more classes and/or spend less 
time working. 
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Chapter 3 


Data and Methods 


This study employs an experimental design methodology to evaluate the 
effectiveness of eight developmental summer bridge programs offered in Texas in the 
summer of 2009. Although the programs varied in their delivery and implementation, they 
all contained four common features — an accelerated format, a college knowledge 
component, academic support, and a stipend. Their common design features made these 
programs sufficiently similar to permit pooling of data across all eight sites for analysis. 
Our site visits during the summer of 2009 allowed us to verify the presence of these four 
features, giving us confidence that the programs were substantially similar. 

This chapter details the methodology used in conducting this study. We first outline 
our general research design and describe the student sample that participated. Then, we 
enumerate the data sources used for analysis and explain our statistical approach to the data 
analysis. It should be noted that implementation research was an important part of the 
overall study, allowing us to learn how developmental summer bridge programs were 
designed and deployed and to assure that there was fidelity to the intended program design. 
The interim report on this research (Wathington et al., 2011) offers our findings on the 
implementation of these eight developmental summer bridge programs. 


Research Design 

Beginning in the winter of 2008-09, students were recruited at eight participating 
institutions throughout the state of Texas. Eligible students were those whose scores on a 
college placement test (which was usually administered by a college in a high school) 
indicated the need for remediation in a subject offered by the college’s summer bridge 
program. At most institutions, only graduating high school seniors were eligible, though at 
South Texas College, returning adults could also participate. 

Eligible students who applied for admission into a developmental summer bridge 
program and agreed to participate in the study were included in the research sample. After 
consenting to participate and completing a baseline intake form, these students were 
randomly assigned to either the program or the control group. About 60 percent of the 
students were assigned to the program group and given the opportunity to take one of the 
available slots in the summer bridge program (793 students), while about 40 percent were 
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assigned to the control group and were able to participate in other college services but were 
not admitted to the program itself (525 students). 1 Students in both groups consented to 
release their program, enrollment, and transcript data to the researchers for two full 
academic years — from the summer of 2009, when the program was held, through the 
spring of 201 1. 

Because students were randomly assigned to the program and control groups, any 
difference between the groups’ average outcomes can be interpreted as the result of being 
permitted to participate in the program (the average treatment effect). Unlike alternative 
methodologies often used to evaluate summer bridge programs and similar interventions, 
the experimental design of this study eliminates concerns of selection bias in the program 
group. For example, an evaluation that compared the outcomes of students who participated 
in a developmental summer bridge program with the outcomes of other students at the 
college who did not would not be able to account for unobservable differences in student 
characteristics. It is possible that, in this scenario, the summer bridge students could be 
more motivated and higher in ability than the nonparticipating students, leading to better 
average outcomes that were the result not of the intervention but of differences between the 
two groups. In an experimental design, the program and control groups are alike on both 
observable characteristics (e.g., gender, race/ethnicity) and unobservable characteristics 
(e.g., motivation, time management skills), allowing us to draw causal inferences about the 
average treatment effect of the program. 

In order to preserve the experimental design, our study measures differences 
between the average member of the entire program group and the average member of the 
entire control group. Within these groups, individual students did not necessarily have 
identical educational experiences during the summer when the bridge programs were 
offered. Not all students assigned to the program group enrolled in and completed the 
summer bridge program. In addition, while students in the control group were not permitted 
to enroll in the summer bridge program, some opted to participate in other substantive 
educational activities that could have affected their later outcomes. 

Among those selected to participate in the developmental summer bridge programs, 
689 students (86 percent) enrolled. Students assigned to the control group were free to 
pursue any other summer activities, including enrolling in other courses at the colleges 
where the summer bridge programs were held. Enrollment data from the eight participating 
colleges show that, during the summer of 2009, 16 percent of students in the control group 
enrolled in a summer course at the college where they had applied for admission to a 


'Several students were removed from the sample due to missing documentation. This accounts for 
the discrepancy between these numbers and those in Table 1.1. 
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summer bridge program. We can gain further insights into the activities of the control group 
from their responses on an intake form completed prior to random assignment, on which 
students were asked what they expected to do during the summer of 2009 if they did not 
attend the summer bridge program. Of those who responded, 32 percent expected to take 
classes at the summer bridge program college or elsewhere, and 74 percent expected to find 
a summer job (some planned to do both). Only 6 percent of students expected to do neither. 


Sample Characteristics 

Student baseline data collected at the eight colleges participating in this study show 
that the overall demographic and educational characteristics of the program and control 
groups were very similar, as expected following random assignment. An omnibus test for 
significant differences between the two groups found that there were no systematic 
differences between program and control group students on the set of background 
characteristics collected, confirming that the random assignment process successfully 
created two comparable groups of students. Table 3.1 shows the characteristics of students 
in the sample. 

The students in the summer bridge programs were predominately Hispanic and 
recent graduates of high school. However, students’ demographic characteristics varied by 
college. In terms of race and ethnicity, El Paso Community College, San Antonio College, 
Palo Alto College, South Texas College, and Texas A&M International University served 
more than 90 percent Hispanic students. At both Lone Star College-Kingwood and Lone 
Star College-CyLair, fewer than half of students were Hispanic (the balance was comprised 
of White, African American, and small numbers of Asian American students). All 
institutions served more females than males, with Palo Alto College having the most 
imbalanced gender ratio, at almost 3:1. 

Students were willing to spend long hours in the summer studying to prepare 
themselves to enter college in the fall. (As one said, “I want to attend so I can take college 
classes and not remedial courses and waste money.”) Responses on the intake form suggest 
that the recruited sample of students was highly motivated despite their remedial need. 
Nearly a quarter of students had previously attempted college credits (through dual 
enrollment or otherwise). All students intended to attend college the following fall, and 
more than three quarters intended to do so full time. 
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Texas Developmental Summer Bridge Programs 
Table 3.1 


Demographics of Full Sample at Baseline 


Characteristic 

% 

Gender 

Female 

62.4 

Male 

37.6 

Age as of June 1, 2009 (Mean = 19.0) 

18 or under 

84.7 

19 or 20 

11.2 

21 or over 

4.1 

Race/ethnicity 

African American 

6.6 

Hispanic 

84.3 

White 

8.7 

Other 

1.8 

Aspirations 

High school 

12.4 

Some college 

1.4 

Certificate 

1.5 

Associate degree 

10.5 

Bachelor’s degree 

34.8 

Master’s degree 

34.6 

Doctoral or professional degree 

3.9 

Previously attempted college credits 

24.2 

Previously earned college credits 

21.2 

Date of high school graduation (among graduates) 

2007 or earlier 

4.4 

2008 

2.7 

2009 

92.8 

Fall 2009 college attendance plans 

Full-time (4 courses per term or more) 

77.6 

Part-time (3 courses per term or less) 

22.4 

Texas Success Initiative (TSI) status 

Met math standards 

20.5 

Met reading standards 

17.7 

Met writing standards 

38.9 


SOURCE: Self-reported by students on the Baseline Information Form. 


NOTES: Sample includes 1,318 students. TSI status is determined by a student’s scores on one of several placement 
tests (i.e., ASSET, COMPASS, ACCUPLACER, and THEA) and indicates readiness to enroll in freshman-level 
academic coursework. While institutions can set higher minimum passing scores than the state, this report relies on 
the cut scores established by Texas. There were no statistically significant differences between the program and 
control group students in any of the three areas (math, reading, or writing) at baseline. 
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Data Sources 

Several data sources were used for the analyses of impact presented in this report, 
including two that were not available when we published our first report on the results of 
this research. 2 The outcomes of students participating in the research were tracked through 
the end of the spring 2011 semester, allowing for two full years of follow-up. 

The Baseline Information Form (BIF), a student background questionnaire 
administered prior to random assignment, was used to ensure that the program and control 
groups were similar on baseline characteristics. The BIF was also used to create 
subgroups based on certain demographic characteristics — namely, gender and mother’s 
highest level of education. Subgroups were analyzed separately to determine whether 
programs had different effects for men and women or for students with parents with 
different levels of education. 

Institutional administrative data provided to NCPR by participating program sites 
was our primary data source for most student outcomes. Institutional data was available for 
all eight of our program sites (El Paso Community College, Lone Star College-CyFair, 
Lone Star College-Kingwood, Palo Alto College, San Antonio College, Saint Philip’s 
College, South Texas College, and Texas A&M International University), as well as the rest 
of the Lone Star College System and two additional colleges that served a large number of 
students in our sample (Laredo Community College and the University of Texas at El 
Paso). Prom these colleges, we received complete student transcripts as well as additional 
data on enrollment, financial aid receipt, graduation, and placement test scores. These 
institutional records were our only data sources for several key measures, including total 
credits earned and financial aid received. 

Texas Higher Education Coordinating Board data, which contained student-level 
information from all Texas colleges, provided a secondary source of information for many 
of our variables. An advantage of the THECB records is that they provided information on 
students in our sample who did not attend one of the colleges for which we have 
institutional records. Though the state of Texas did not yet collect transcript records during 
the time we tracked our students, the THECB data did contain detailed information on 
students’ developmental progression and first college-level courses in math, reading, and 
English. In this study, we also used the THECB definitions of the courses considered to be 
first college -level courses in each subject area. Additionally, the data provided information 
on enrollment, credits attempted, and graduation. 


2 

"The two new sources of data were the National Student Clearinghouse data and student survey data 
obtained using a follow-up telephone and email survey. 
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National Student Clearinghouse data, containing enrollment and graduation 
information from more than 3,300 colleges around the country, was used as an additional 
source. We anticipated that a significant number of students might enroll in private or out- 
of-state colleges whose data would not be available from any of our other sources. 
However, less than 1 percent of the sample was found to enroll at such colleges. 

Follow-up surveys were administered only to students who did not appear in any of 
our records as having enrolled in college as of spring 2010. Surveys were conducted via 
telephone and email; respondents included students participating in the research or their 
relatives. Data from these surveys supplemented our other sources of information and 
allowed us to learn more about what students who did not continue in college were doing 
instead. The research firm HumRRO was retained to conduct this survey. A total of 332 
students were targeted, and 213 were successfully surveyed (a 64 percent response rate). 

Because we utilized several data sources in constructing our dataset for analysis, we 
had to make decisions on how to prioritize the different data sources when we encountered 
discrepancies or missing information. 2 * 4 In some of the more prominent areas of discrepancy, 
we tested alternate methods of data prioritization and confirmed that prioritizing data 
sources in a different way did not significantly affect the results. The most important of 
these decisions and alternate analyses are noted in the text and tables throughout this report. 


Overview of the Data Analysis 

In comparing outcomes for the program and control groups, we focused on a limited 
number of primary outcomes and several secondary outcomes. 5 The primary outcomes 


2 

Out of 154 students who were never found to be enrolled in college through one of our other data 
sources, 96 were successfully surveyed. Of these, 26 students reported being enrolled at some time during 
our tracking period. 

4 In general, for outcomes that were available from a limited number of data sources, students missing 
information from that data source were coded with “0” rather than missing. For example, since we only 
had information on earned credits from the institutional records, a student whose data was not available 
from these institutions would be considered to have earned 0 credits, even if records from the THECB or 
the National Student Clearinghouse suggested that the student may have been enrolled elsewhere. Though 
this might reflect itself as an underestimate of the baseline levels of credits earned, there is no reason to 
think that program and control group students would be disproportionately affected, and therefore there 
should be no bias in the analysis. If we had only analyzed these outcomes based on students with non- 
missing values, our results might be biased because the program and control groups could be 
disproportionately enrolled (and therefore have disproportionately missing values for these outcomes). 

5 We focus on a smaller set of primary outcomes in order to avoid bias arising from multiple 
hypothesis testing. When hypothesis tests are conducted on multiple outcomes with multiple subgroups of 
students and/or schools, it increases the likelihood of concluding that some findings are statistically 
significant when in fact they are not (resulting in a “Type I” error or a “false positive”). 
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include a measure of academic persistence, total college-level credits earned, and 
completion of college-level courses in math, reading, and writing. All other outcomes were 
considered secondary. 

Given that this was a randomized controlled experiment, the model we employed to 
estimate impacts was fairly simple. Our analysis was accomplished by fitting the following 
impact model to the analysis samples: 

(i) 

K 

In this model, F, is the outcome of interest for student i. Tj is an indicator of 
treatment group membership (treatment status) equal to 1 if student i was assigned to the 
summer bridge program and zero otherwise. C ki is a dummy equal to 1 if student i is at 
campus k and zero otherwise. & is an error term. J3 0 , therefore, is the estimated impact of 
the program on outcome F. 

J3 Q is an estimate of the impact of the developmental summer bridge program for the 
average student in the analysis sample. The students were not selected to be a random 
sample of a larger population of sites, and the impact estimates are not generalizable to a 
larger population of colleges or students. Indicators for each college (C k ) are included in the 
model because random assignment was conducted separately for each college and because 
including these indicators allows us to obtain more precise estimates by reducing the 
influence of variation across colleges. 


Limitations 

Our study design and data availability limit what we can say about our findings in 
several ways. First, in order to reach the sample size required to detect minimum effects, we 
had to pool data across the sites. This research was not designed to detect effects at the level 
of the individual participating college. 

Second, like most research designs that estimate program effects, the random 
assignment design evaluates the overall impact of the developmental summer bridge 
program but cannot determine if certain program components are more effective than 
others. For example, we cannot separate the effect of the student stipend from the effect of 
the course work provided. 

Third, for each outcome, we conducted analyses using the best data available. For 
two important outcomes — total college credits earned and financial aid receipt — we were 
only able to examine outcomes for students who attended college at one of the ten 
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institutions from which we obtained data directly. For students who enrolled at other 
colleges, no data were available for these outcomes. Additionally, we found some 
discrepancies in the data from different sources for the same students, forcing us to make 
decisions about how to prioritize our data sources. The existence of such discrepancies 
suggests that there may be substantial “measurement error” in our data, though as noted 
above, we confirmed that alternate methods of prioritizing data sources did not significantly 
affect our results. 

Fourth, we would ideally be able to compare pre -random assignment and post- 
random assignment results on placement tests, but we were unable to obtain these for our 
entire sample. Therefore, we could not precisely determine the impact of assignment to a 
developmental summer bridge program on placement into math, reading, and writing in the 
fall. Instead of directly measuring placement, we measured the impact of the program on 
the courses that students attempted and passed. We primarily focused on students’ 
completion of their first college-level courses in math, reading, and writing because these 
outcomes are well defined for the entire sample (regardless of college enrollment), and 
therefore we were able to examine these outcomes causally. We additionally performed an 
exploratory and non-experimental analysis on students’ progression through the 
developmental sequence in each of these three subjects. 

Fifth, the results obtained through this study may be limited in their external 
validity and may not be applicable to all summer bridge programs. The programs we 
studied were located in a single state and targeted a population with specific developmental 
needs. Programs that target different populations and programs at colleges that offer a 
different experience for the control group (such as more or less intensive alternative 
program offerings) may see different results. 

Despite these limitations, our use of experimental design should inspire confidence 
in the results of this study. Because of its rigorous design, this study provides the strongest 
evidence to date on the effectiveness of developmental summer bridge programs. 
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Chapter 4 


Impact Findings 

The primary purpose of this chapter is to examine whether participation in 
developmental summer bridge programs led to better student outcomes in terms of credit 
accumulation, persistence, and completion of students’ first college-level courses. This 
chapter presents impact findings at the end of the two years following random assignment, 
and it includes estimates for each semester of the two-year follow-up period. After a brief 
overview of the main findings, the chapter describes the measures used to evaluate the 
program and presents the impact findings in more detail. This chapter also explores whether 
program students experienced a reduced need for remediation in college, whether impacts 
varied between subgroups, and whether the program influenced outcomes related to 
financial aid. 

After two years of follow-up, these are the main findings of this study: 

• The programs had no effect on the average number of credits attempted 
or earned. Program group and control group students attempted the 
same number of credits (30.3). Students in the program group earned an 
average of 19.4 credits, and students in the control group earned an 
average of 19.9 credits; the difference in their outcomes is not 
statistically significant. 

• The programs had an impact on first college-level course completion in 
math and writing that was evident in the year and a half following the 
program but no impact on first college-level course completion in 
reading during this same period. On average, students in the program 
group passed their first college-level math and writing courses at higher 
rates than students in the control group during this period. By the end 
of the two-year follow-up period, however, the differences between the 
two groups are no longer statistically significant. 

• There is no evidence that the programs impacted persistence. During the 
two-year follow-up period, students in the program group enrolled in an 
average of 3.3 semesters, and students in the control group enrolled in an 
average of 3.4 semesters, a difference that is not statistically significant. 
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Impact Estimates for Primary Outcomes 

Table 4.1 presents impact estimates two years after random assignment for key 
outcomes central to the program’s theory of change. Box 4.1 discusses how to read the 
tables in this report. 


Texas Developmental Summer Bridge Programs 
Table 4.1 

Key Academic Outcomes Two Years After Random Assignment 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Semesters registered at any college 11 

3.3 

3.4 

-0.1 

.37 

0.1 

Total credits attempted 11 

30.3 

30.3 

0.0 

.98 

1.2 

College-level 

24.2 

23.5 

0.7 

.54 

1.1 

Developmental 

6.1 

6.7 

-0.6* 

.09 

0.4 

Total credits earned 

19.4 

19.9 

-0.5 

.59 

1.0 

College-level 

15.9 

15.9 

0.0 

.97 

0.9 

Developmental 

3.5 

4.0 

-0.6** 

.03 

0.3 

Passed first college-level math course (%) 

46.5 

43.0 

3.5 

.19 

2.7 

Passed first college-level reading course (%) 

72.6 

71.6 

1.0 

.68 

2.4 

Passed first college-level writing course (%) 

71.7 

68.3 

3.3 

.18 

2.5 

Sample size (total = 1,318) 

793 

525 





SOURCE: NCPR calculations from Texas Higher Education Coordinating Board data and college transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. A two-tailed f-test was applied to 
differences between research groups. Estimates are adjusted by site. 

*p < .10. **p < .05. ***p < .01. 

“The number of semesters registered at any college is calculated based on Texas Higher Education Coordinating 
Board data, college transcript data. National Student Clearinghouse data, and student survey data. 
b Credits attempted and earned are calculated based on college transcript data and include summer 2009 and 
summer 2010 terms. 
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Box 4.1 


How to Read the Impact Tables in This Report 

Many tables in this report use a similar format. The example table below displays impacts on two 
academic measures. The first row. for example, shows that 10.7 percent of the program group 
members and 4.8 percent of the control group members passed the first college-level math course. 

Because individuals were assigned randomly either to the program group or to the control group, 
the effects of the program can be estimated by the difference in outcomes between the two groups. 
The column labeled “Difference (Impact)” shows the differences between the two research 
groups’ outcomes — that is, the program’s estimated impacts on the outcomes. For example, the 
estimated impact on passing the first college-level math course can be calculated by subtracting 
4.8 percent from 10.7 percent, yielding an increase or estimated impact of 5.9 percentage points 
(rounded). The term impact refers to the “added value” of the program, or the program’s effects 
that go above and beyond the effects of the services provided to the control group. This difference 
represents the estimated impact rather than the true impact because, although study participants 
are randomly assigned to the program and control groups, there is still a possibility that differences 
could be observed by chance. 

Differences marked with one or more asterisks are statistically significant, meaning that there is 
only a small probability that the observed difference occurred by chance. The number of asterisks 
indicates the probability of observing differences at least as extreme as the observed differences if 
the program’s true impact is zero. One asterisk corresponds to a 10 percent probability; two 
asterisks, a 5 percent probability; and three asterisks, a 1 percent probability. For example, as the 
first row of the example table shows, the program’s estimated impact on students passing the first 
college-level math course is 5.9 percentage points. The three asterisks indicate that this difference 
is statistically significant at the 1 percent level, meaning that there is less than a 1 percent chance 
of observing a difference this large if the program’s true impact is zero. In other words, one can be 
99 percent confident that the program had a positive impact on students passing the first college- 
level math course. 

The statistical significance is calculated using the standard error of the impact estimate, shown in 
the rightmost column. The standard error is a measure of uncertainty or variability around the 
impact estimate. There is about a 90 percent chance that the true impact is within plus or minus 
1.65 standard errors of the estimated impact, roughly a 95 percent chance that the true impact is 
within plus or minus 1 .96 standard errors of the estimated impact, and about a 99 percent chance 
that the true impact is within plus or minus 2.58 standard errors of the estimated impact. For 
example, in the first row of the table below, there is roughly a 99 percent chance that the 
program’s impact on students’ likelihood of passing the first college-level math course lies 
between 2.03 and 9.77 percentage points, calculated as 5.9 ± (2.58 x 1.5). 


Outcome 

Program 

Control 

Difference 

Standard 

Group (%) 

Group (%) 

(Impact) 

Error 

Passed first college-level math course as of Fall 2009 

10.7 

4.8 


1.5 

Passed first college-level reading course as of Fall 2009 

32.1 

28.8 

3.3 

2.4 
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Persistence 

Persistence toward the completion of a credential is of major concern to those 
involved in higher education. Participation in developmental summer bridge programs 
could cause students to have a successful initial experience with college, which may 
encourage them to persist. Persistence is measured by the cumulative number of semesters 
that students enrolled in college over the two-year period of the study. 

There is no evidence that the program impacted persistence in college. Table 4.1 
shows that, as of the end of the follow-up period, students in the program group enrolled in 
an average of 3.3 semesters, while students in the control group enrolled in an average of 
3.4 semesters; the difference in their outcomes is not statistically significant. Table 4.2 
shows impact estimates for persistence and enrollment for each semester of the two-year 
follow-up period after the summer 2009 program semester. The first row for each semester 
shows the percentage of students in the program and control groups who registered at any 
college. The second row for each semester shows the cumulative average number of 
semesters that students in each group registered up to that time. 

In the fall 2009 semester, 81.6 percent of students in the program group registered 
at a college, compared with 80.6 percent of students in the control group. The difference is 
not statistically significant. Consequently, there is no evidence that the program impacted 
enrollment in the semester immediately following the summer bridge programs. In general, 
the programs do not appear to have impacted registration rates in other semesters during the 
two-year follow-up period either, although a statistically significant difference does arise in 
the summer of 2010. This, however, could simply be due to chance because, all else being 
equal, the likelihood of observing a statistically significant impact increases as more 
semesters and more outcomes are considered. The magnitude of the summer 2010 estimate 
also appears to be an anomaly. 

The semester-by-semester estimates for persistence parallel this trend. As of the fall 
2009 semester, both groups of students had enrolled for an average of 0.8 cumulative 
semesters. The cumulative number of semesters enrolled grows for each subsequent 
semester and reflects a running total. For every semester, the difference between program 
and control group students in the cumulative semesters registered remains close to zero, and 
at no point is the difference statistically significant. In short, the program does not appear to 
have meaningfully impacted registration or persistence at any point in the follow-up period. 


6 The cumulative number of semesters is estimated using transcript data from the colleges, data from 
the THECB, and data from the National Student Clearinghouse. Data from summer 2009 are not included 
in this measure. Including them does not significantly change the two-year findings. The summer 2010 
session is counted as distinct. Due to data limitations, winter sessions are included with spring semesters. 
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Texas Developmental Summer Bridge Programs 
Table 4.2 


Persistence and Enrollment Outcomes by Semester 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Fall 2009 






Registered at any college (%) 

81.6 

80.6 

0.9 

.67 

2.2 

Total semesters registered at any college 

0.8 

0.8 

0.0 

.67 

0.0 

Sorine 2010 






Registered at any college (%) 

79.1 

79.5 

-0.4 

.86 

2.3 

Total semesters registered at any college 

1.6 

1.6 

0.0 

.89 

0.0 

Summer 2010 






Registered at any college (%) 

39.6 

45.5 

-5.9** 

.03 

2.7 

Total semesters registered at any college 

2.0 

2.1 

-0.1 

.33 

0.1 

Fall 2010 






Registered at any college (%) 

66.8 

67.7 

-0.9 

.74 

2.6 

Total semesters registered at any college 

2.7 

2.7 

-0.1 

.39 

0.1 

Soring 2011 






Registered at any college (%) 

63.9 

65.8 

-1.9 

.49 

2.7 

Total semesters registered at any college 

3.3 

3.4 

-0.1 

.37 

0.1 

Sample size (total = 1,318) 

793 

525 





SOURCE: NCPR calculations from Texas Higher Education Coordinating Board data, college transcript data. 
National Student Clearinghouse data, and student survey data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. A two-tailed f-test was applied to 
differences between research groups. Estimates are adjusted by site. 

*p < .10. **p < .05. ***p < .01. 
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Credit Accumulation 

Credit accumulation serves as a rough proxy for progress toward a degree. 
Compared with students who have similar developmental education needs, students who 
earn more developmental credits are generally making greater progress toward completing 
developmental course sequences, and students who earn more college-level credits are 
generally moving closer to satisfying degree requirements. 7 8 This evaluation considers the 
total number of credits attempted and earned during the follow-up period, including 

g 

estimates for both college-level (i.e., degree-applicable) and developmental credits. 

As Table 4.1 shows, there is no evidence that the program impacted attempted 
credits or earned total credits on average. By the end of the spring 2011 semester, students 
in the program and control groups had attempted an average of 30.3 credits (these include 
both college-level and developmental credits). Students in the program group earned an 
average of 19.4 credits, and students in the control group earned an average of 19.9 credits, 
a difference that is not statistically significant. These estimates include students who did not 
earn any credits. 

There is also no statistically significant difference in the estimated average number 
of attempted college-level credits (24.2 for students in the program group and 23.5 for 
students in the control group) or the average number of college-level credits earned (15.9 
for both groups). 

For developmental credits, the estimated difference is statistically significant on 
both measures. On average, program students attempted 0.6 fewer developmental education 
credits than students in the control group and also earned an average of 0.6 fewer 
developmental credits. Further analysis, however, suggests that these differences do not 
reflect real differences in the amount of developmental classroom time in which students in 
the two groups participated. Instead, several colleges’ developmental summer bridge 
programs involved freestanding developmental education work for which students did not 
receive credit. Subgroup analyses show that program students who enrolled in these 
colleges earned fewer developmental credits than their counterparts in the control group, 
even though they participated in developmental work through the program; they simply did 
not earn credit for this work. At colleges where course-based summer bridge programs did 


7 A11 of the students in this study were deemed unprepared for college-level courses in at least one 
subject, so very few students were expected to earn two-year degrees within the two-year period. Results 
indicate that 3.3 percent of the program group and 2.7 percent of the control group earned degrees (the 

difference is not statistically significant). 

8 

All estimates related to credits attempted and earned are derived from transcript data from the 
colleges. The THECB provided data on all attempted credits but only on developmental education credits 
earned. Including the available THECB data does not substantively change the findings. 
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award credit for developmental education work, program and control group students earned 
about the same number of developmental education credits. Consequently, the overall 
difference in developmental credits earned appears to be driven by colleges with 
freestanding programs that did not award credit for developmental education classroom 
time; the data do not suggest that the program reduced the amount of developmental work 
that students completed (see Table A. 7 for more detail). 

Table 4.3 shows cumulative impact estimates for attempted credits and earned credits 
for each semester of the two-year follow-up period after the summer 2009 program semester. 
In general, there do not appear to meaningful differences in credits attempted or credits 
earned at any point during the two-year follow-up period. Although statistically significant 
differences do arise for developmental credits, they do not appear to reflect meaningful 
differences in the amount of developmental work attempted and completed by students. 

College-Level Course Completion 

We also examine whether the developmental summer bridge programs helped 
students advance to and complete their first college-level courses in math, reading, and 
writing. This indicates whether program group students were less likely to need remediation 
in college or had advanced more rapidly through the developmental course sequence. 

The last three rows in Table 4.1 show whether the program helped students advance 
into and through first college-level courses in math, reading and writing. These rows 
indicate the percentage of students in the program and control groups who passed first 
college-level courses in each subject up to two years after random assignment. 9 Although 
the program group does initially show small gains over the control group (discussed below), 
at the end of the two-year follow-up period, none of the differences in their outcomes are 
statistically significant. After two years, 46.5 percent of students in the program group 
passed college-level math, compared with 43.0 percent of students in the control group. In 
reading, 72.6 percent of students in the program group passed a college-level course, 
compared with 71.6 percent in the control group. In writing, 71.7 percent of students in the 
program passed a college-level course, compared with 68.3 percent of students in the 
control group. 


^Transcript data from the colleges and the THECB data provide information about college-level 
course completion. For this report, students are determined to have passed a college-level course in a 
given subject if either data source identifies them as passing the course in any semester. Considering the 
data sources separately or prioritizing one data source over the other does not significantly change the 
substantive conclusions. 
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Texas Developmental Summer Bridge Programs 
Table 4.3 


Persistence and Enrollment Outcomes 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Fall 2009 

Total credits attempted 

9.7 

9.4 

0.3 

.45 

0.4 

College-level 

6.2 

5.7 

0.4 

.16 

0.3 

Developmental 

3.5 

3.7 

-0.2 

.44 

0.2 

Total credits earned 

6.1 

6.2 

-0.1 

.68 

0.3 

College-level 

4.0 

4.0 

0.1 

.74 

0.3 

Developmental 

2.0 

2.3 

-0.2 

.20 

0.2 

Surine 2010 

Total credits attempted 

17.4 

17.0 

0.4 

.52 

0.6 

College-level 

12.4 

11.6 

0.8 

.13 

0.5 

Developmental 

5.0 

5.4 

-0.4 

.16 

0.3 

Total credits earned 

10.8 

11.0 

-0.2 

.76 

0.5 

College-level 

8.0 

7.8 

0.2 

.66 

0.5 

Developmental 

2.9 

3.2 

-0.4* 

.10 

0.2 

Summer 2010 

Total credits attempted 

19.1 

19.0 

0.1 

.85 

0.7 

College-level 

13.9 

13.4 

0.5 

.41 

0.6 

Developmental 

5.2 

5.6 

-0.4 

.22 

0.3 

Total credits earned 

12.0 

12.5 

-0.5 

.41 

0.6 

College-level 

9.0 

9.2 

-0.1 

.81 

0.5 

Developmental 

3.0 

3.4 

-0.4 

.11 

0.2 

Fall 2010 

Total credits attempted 

25.2 

24.8 

0.4 

.66 

1.0 

College-level 

19.4 

18.5 

0.9 

.28 

0.9 

Developmental 

5.8 

6.3 

-0.5 

.14 

0.3 

Total credits earned 

16.0 

16.4 

-0.4 

.64 

0.8 

College-level 

12.7 

12.5 

0.2 

.81 

0.7 

Developmental 

3.3 

3.8 

-0.5** 

.03 

0.3 

Sprint! 2011 

Total credits attempted 

30.3 

30.3 

0.0 

.98 

1.2 

College-level 

24.2 

23.5 

0.7 

.54 

1.1 

Developmental 

6.1 

6.7 

-0.6* 

.09 

0.4 

Total credits earned 

19.4 

19.9 

-0.5 

.59 

1.0 

College-level 

15.9 

15.9 

0.0 

.97 

0.9 

Developmental 

3.5 

4.0 

-0.6** 

.03 

0.3 

Sample size (total = 1,318) 

793 

525 





SOURCE: NCPR calculations from college transcript data. 


NOTES: Rounding may cause slight discrepancies in sums and differences. A two-tailed f-test was applied to 
differences between research groups. Estimates are adjusted by site. 

*p < .10. **p < .05. ***p < .01. 
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First College-Level Course Completion in Math 

Figure 4.i compares the cumulative percentage of students in the program and 
control groups who passed the first college-level math course in each semester of the 
follow-up period. There is strong evidence that the program positively impacted the 
percentage of students who passed the first college-level math course in the semester 
following the developmental summer bridge program, and the impact is evident for the first 
year and a half. In fall 2009, 10.7 percent of students in the program group passed college- 
level math, compared with 4.8 percent of students in the control group — a statistically 
significant difference of 5.9 percentage points. In the spring of 2010, the estimated 
difference grows to 9.4 percentage points and remains statistically significant. In 
subsequent semesters, however, the estimated difference steadily decreases, although it 
remains statistically significant through the fall of 2010. By the end of the follow-up period 
in the spring of 201 1, the difference is no longer statistically significant. 

First College-Level Course Completion in Reading 

Figure 4.2 shows the percentage of students in each group who passed their first 
college-level reading course in each semester of the follow-up period. In every semester, 
students in the program group passed college-level reading at slightly higher rates than 
students in the control group, but the difference is only statistically significant (at the 10- 
percent level) in the spring of 2010. It is difficult to tell whether this marks a real impact or a 
difference due to chance. The pattern of positive differences is suggestive, but conclusions 
about impacts on college-level reading are more tentative than those for math. By the end of 
the follow-up period, the estimated difference decreases to one percentage point and is not 
statistically significant, suggesting the program had no impact on completion of the first 
college -level reading course at the end of two years. 

First College-Level Course Completion in Writing 

Figure 4.3 shows the percentage of students in each group who passed the first 
college-level writing course in each semester of the follow-up period. There is strong 
evidence that the program positively impacted the percentage of students who passed the 
first college-level writing course in the semester following the developmental summer 
bridge program. Figure 4.3 shows that 32.9 percent of students in the program group passed 
college-level writing in the fall of 2009, compared with 28.8 percent of students in the 
control group, and the estimated difference of 4.1 percentage points is statistically 
significant at the 10-percent level. The estimated difference is positive throughout the 
follow-up period and statistically significant for the first four semesters, but as with math, 
by the end of the final semester, the difference is no longer statistically significant. 
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Texas Developmental Summer Bridge Programs 
Figure 4.1 

Cumulative Percentage of Students Passing College-Level Math by Semester 
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SOURCE: NCPR calculations from Texas Higher Education Coordinating Board data and college transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. A two-tailed t - test was applied to 
differences between research groups. Estimates are adjusted by site. 

< .10. **p < .05. ***/? < .01. 
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Texas Developmental Summer Bridge Programs 
Figure 4.2 

Cumulative Percentage of Students Passing College-Level Reading by Semester 
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SOURCE: NCPR calculations from Texas Higher Education Coordinating Board data and college transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. A two-tailed f-test was applied to 
differences between research groups. Estimates are adjusted by site. 

< .10. **p < .05. ***p < .01. 
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Texas Developmental Summer Bridge Programs 
Figure 4.3 

Cumulative Percentage of Students Passing College-Level Writing by Semester 
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SOURCE: NCPR calculations from Texas Higher Education Coordinating Board data and college transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. A two-tailed t - test was applied to 
differences between research groups. Estimates are adjusted by site. 

< .10. **p < .05. ***p < .01. 
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Secondary Measures of Program Effects 

In addition to the primary outcome measures discussed above, this evaluation 
examines students’ progression though the developmental sequences, students’ receipt of 
financial aid, and variation in program impacts for different student subgroups. 

Progression Through Developmental Education Sequences 

To examine whether program and control group students progressed through the 
developmental education sequences at different rates, we performed an analysis of the 
highest level that students completed in math, reading, and writing. 10 Interpretation of this 
analysis is complicated because not all students in the sample attempted courses in each 
subject. Generally, around 85 percent of students attempted at least one course in a given 
subject by the end of the follow-up period. Because the program and control group 
students who attempted each subject may have differed systematically from each other 
and from the larger sample, estimates pertaining to students’ progression should not be 
interpreted causally. 

Table A.l (located in Appendix A) presents data on the percentages of students in 
the program and control groups who attempted each subject and compares the groups’ 
average highest level completed in each subject. The indices in this table are measures of 
how far students who attempted each subject area had progressed, on average, by the end of 
the two-year follow-up period. This information is shown on a scale of 0 to 3, where 0 is 
college level and 3 is three levels below college-level. For example, program group students 
who attempted math were, on average, 0.8 courses below college-level in math at the end of 
the two-year period. Control group students who attempted math were 1.0 course below 
college-level, a little lower in the math sequence. This difference was not statistically 
significant. Both groups of students were, on average, just below college-level in reading 
and writing at the end of the two-year period. 

Financial Aid 

As discussed in Chapter 1, the college knowledge component of the developmental 
summer bridge programs included information about financial aid. At the Alamo Colleges 
(San Antonio, Palo Alto, and St. Philip’s), in particular, students received assistance in 
filling out financial aid application forms when they enrolled in the program. Consequently, 
we examined whether program group students received more financial aid than control 


"’Transcript data from the colleges and the THECB data provide information about college -level 
course completion. For this report, students are determined to have passed courses in a given subject if 
they were identified as doing so by either data source. 
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group students and whether differences are evident for the students in the program group at 
the Alamo Colleges. Table 4.4 shows the estimated impacts on financial aid. 

The program positively impacted the percentage of students who received financial 
aid during the fall semester following the summer program: 59.2 percent of students in the 
program group received some form of financial aid, compared with 54.1 percent of students 
in the control group. The 5.1 percentage point impact estimate is statistically significant at 
the 10-percent level and appears evenly distributed across the receipt of Pell grants (3.4 
percentage point estimate) and other aid (3.1 percentage point estimate). There is no 
evidence that program group students received greater amounts of aid on average than 
control group students. 

What is more, there is no evidence that program students continued to receive more 
financial aid after the fall semester following the summer program (see Table A.3). 
Subgroup analyses comparing Alamo Colleges to the other colleges provide a potential 
explanation. There is some evidence that the impact in financial aid is related to how the 
program was implemented at the Alamo Colleges, where students appear to have received 
more intensive assistance with completing their financial aid forms. The impacts on 
financial aid receipt were significantly larger at the Alamo Colleges than at the other 
colleges; this appears to be driving the average impact on financial aid receipt (see Table 
A.4). Because these impacts are evident only during the first year and only at the Alamo 
Colleges, it may be that intense assistance is necessary to produce an effect. 

Subgroup Analyses 

Additional analyses examined whether student outcomes varied in relation to factors 
thought to be predictive of student success. Demographic characteristics examined included 
gender and socioeconomic status as indicated by mother’s highest level of education. “We 
also examined whether course-based and freestanding developmental summer bridge 
programs produced different impacts. There is no evidence that program impacts varied by 
any of these characteristics (see the Tables A.5-A.7 for detailed estimates). 


1 'Financial aid data provided by the colleges are used to examine this measure. 
l2 See Ensminger et al. (2000) for more detail about this measure. 
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Texas Developmental Summer Bridge Programs 
Table 4.4 

Financial Aid Outcomes in the 2009-10 Academic Year 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Fall 2009 

Registered at summer bridge school (%) 

71.6 

70.1 

1.5 

.56 

2.5 

Received financial aid (%) 

59.2 

54.1 

5.1* 

.06 

2.8 

Pell grant 

47.0 

43.7 

3.4 

.22 

2.8 

Other aid 

49.7 

46.6 

3.1 

.27 

2.8 

Average aid received ($) 

1,053 

1,008 

45.4 

.48 

63.9 

Pell grant 

954 

899 

55.0 

.39 

63.8 

Other aid 

355 

370 

-15.3 

.63 

32.0 

Sample size (total = 1,318) 

793 

525 




Spring 2010 

Registered at summer bridge school (%) 

66.7 

64.6 

2.1 

.41 

2.6 

Received financial aid (%) 

58.4 

55.0 

3.4 

.22 

2.7 

Pell grant 

46.8 

45.2 

1.6 

.57 

2.7 

Other aid 

50.5 

49.5 

1.0 

.73 

2.8 

Average aid received ($) 

1,014 

1,002 

12.5 

.84 

62.5 

Pell grant 

918 

897 

21.0 

.74 

62.3 

Other aid 

404 

416 

-11.3 

.73 

32.8 

Sample size (total = 1,050) 

525 

525 





SOURCE: NCPR calculations from college transcript and financial aid data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. Aid amounts are rounded to the nearest 
dollar. A two-tailed ?-test was applied to differences between research groups. Estimates are adjusted by site. 

*p < .10. **p < .05. ***p < .01. 
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Chapter 5 


Program Costs and Cost Analysis 

The cost analysis of the developmental summer bridge programs was originally 
conducted to provide descriptive information on program costs as well as to conduct a cost- 
benefit or cost-effectiveness analysis. While the analysis of program costs was 
straightforward, we were unable to perform a cost-effectiveness analysis because the 
outcomes with positive impacts (e.g., passing college-level math in the first year and a half 
following the program) are not easily monetized. Instead, we performed a “break-even” 
analysis using a cost-effectiveness analysis framework. In our examination, we considered 
two questions. First, what factors contributed to the variations in the costs of running the 
programs across the eight sites? Second, what impact on college-level credit accumulation 
would the programs have had to produce in order to be considered cost effective? 


Summary of Program Costs 

Program costs were calculated using the ingredients method, which attempts to 
identify and assign a cost to each of an intervention’s many parts (Levin & Me Ewan, 2001). 
Information on costs was collected via a standard questionnaire devised by NCPR for the 
developmental summer bridge programs. 1 2 The costs are summarized costs in Table 5.1. 
None of the colleges applied a direct cost-recovery formula for program overhead, so a 
uniform rate of 30 percent was applied on all direct costs.“ For simplicity, we divided the 
cost ingredients into four areas: staffing, other costs, student resources, and overhead. 

Across the eight sites, approximately one third of the costs were for staffing 
(instructional, managerial, and administrative staff), and just over one quarter were for 
student stipends. The programs’ summer 2009 recruitment budgets were higher than usual 
as a result of the experimental design, which required a higher number of applicants to the 
program than could be served. NCPR encouraged participating colleges to recruit 
vigorously and provided resources and technical assistance for that purpose. 


1 The questionnaire, which was administered to program staff, collected information on ten 
components: participation rates; program duration; teaching staff; non-teaching, administrative, and 
recruitment staff; in-kind resources; course preparation; materials, facilities, and overhead costs; funding 
sources; tuition and fees; and any other expenses. Wage estimates include fringe benefits (if incurred). 
The questionnaire form is available on request. 

2 The 30 percent overhead rate is based on review of overhead rates applied at colleges across the 
U.S. We test for sensitivity to the overhead rate below. 
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Texas Developmental Summer Bridge Programs 
Table 5.1 


Program Costs 



El Paso 

CyFair 

Kingwo 

od 

South 

Texas 

TAMIU 

Palo 

Alto 

San 

Antonio 

St. 

Philip's 

Average 

Program group 
members 

165 

75 

52 

83 

126 

53 

91 

154 

100 

Staffing costs ($) 

504 

222 

393 

418 

726 

278 

587 

168 

426 

Other costs ($) 

68 

183 

425 

165 

570 

365 

97 

175 

235 

Student resources ($) 

414 

238 

424 

451 

512 

357 

-38 

373 

354 

Overhead ($) 

296 

193 

373 

310 

542 

300 

194 

215 

304 

Cost per program 
group member ($) 

1,282 

835 

1,615 

1,343 

2,349 

1,299 

840 

930 

1,319 

Total cost ($) 

211,512 

62,633 

83,963 

111,463 

296,033 

68,853 

76,458 

143,218 

123,160 


NOTE: Costs are reported in 2009 dollars. 


Total costs ranged from $62,633 to $296,033 per college. Part of the variation in 
costs is due to variations in program duration, intensity, and enrollment. The large variation 
in students served is the most important driver of the cost variation. Table 5.1 shows the 
average cost per participant. Across the eight sites, average costs ranged from $835 to 
$2,349. The average across all eight sites was $1,319 (with a standard deviation of $502). 3 
Unsurprisingly, there is no strong evidence of economies of scale in terms of numbers of 
students enrolled; the high-value stipend (up to $400, but averaging $354 for students who 
received a stipend) is a constant for each student. 


Cost Variation 

The costs of running the developmental summer bridge programs varied widely 
across institutions. Examining two institutions with above average costs and two with 
below average costs helps to illuminate which programmatic aspects drove these variations. 
In general, the higher cost programs devoted extra funds to program planning and 
preparation as well as to services for students, and the lower cost programs were able to 
take advantage of existing resources or arrangements that lowered their costs. 


3 Some costs may be interpreted as start-up costs, which are unlikely to be needed if the programs are 
run in subsequent years. Amortizing these costs over three years reduces the average cost of the programs. 
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Higher Cost Programs 

TAMIU’s program had the highest cost per student at $2,349. In fact, removing 
TAMIU from the calculation of cost per student decreases the average by almost 12 
percent, from $1,319 to $1,163 per student. The high cost of TAMIU’s program appears to 
be the result of a number of factors. First, the number of instructional contact hours (100) 
was relatively high. Second, TAMIU offered more benefits to workers than other 
institutions did. As a percentage of salary, TAMIU calculated benefits at 29 percent, 
compared with other sites that either did not offer benefits or utilized a rate of between 12 
percent and 22 percent. Third, TAMIU emphasized the use of tutors; the program employed 
20 tutors for 19 hours per week. Both the number of tutors and the number of hours worked 
per week were higher at TAMIU than at other sites. Fourth, TAMIU provided a daily lunch 
for all students at a cost of approximately $20,000. Although no single expenditure was 
uncharacteristically high, TAMIU was on the higher end of many costs, and this appears to 
have had an incremental, additive effect on overall program costs. 

The program at Lone Star College-Kingwood had the second-highest cost per 
student at $1,615. Its average appears to have been driven up by costs associated with 
program preparation, facilities, and learning resources. Kingwood spent an unusually large 
number of hours preparing to offer the program. This was due to the fact that the program 
was run by a new coordinator and was located in a new department of the college in the 
summer of 2009. Furthermore, much of this work was coordinated by the Dean of Student 
Services, who had a higher salary than the program directors at other colleges. Kingwood 
also provided free lunches for all students and held catered opening and closing ceremonies 
for students and their families (approximately 100 people). 

Lower Cost Programs 

San Antonio College’s cost per student was lower than the average at $840. This 
appears to be due to the $480 students had to pay to participate in the program, which was 
deducted from the final cost to the college. Although a large percentage of participants 
received Pell grants or other financial aid, the revenue that the institution received from 
students significantly decreased the costs required to run the program. San Antonio College 
actually had a small negative cost in the student resources category, which included the 
stipend of up to $400. 

Lone Star College-CyFair had the lowest per-student program costs at $835. This 
appears to be the result of small savings in many areas. First, there were no benefits 
provided to personnel who helped to administer or teach in the program. Second, while 
institutions normally cover the cost of textbooks, CyFair was able to loan students 
textbooks that had been purchased in previous years. Further, students had to pay $150 to 
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attend the program. 4 This effectively reduced the value of the student stipend from $400 to 
$250, as the students’ program tuition was taken directly from the stipend. This reduced the 
costs associated with the student resources category, though not as dramatically as at San 
Antonio College. 


Break-Even Analysis 

For the break-even analysis, the primary outcome of interest is marginal earned 
college credits accumulated by the program group. Other outcomes could be utilized, but they 
each present problems in this type of analysis. For example, academic persistence, defined 
here as the number of semesters enrolled, is not sufficiently distinguishable from credit 
accumulation. Certain other outcomes — such as passing the first college-level course in 
math, reading, or writing — are difficult to monetize. Secondary outcomes, such as financial 
aid receipt, may be valuable to the student, but they are only socially valuable insofar as they 
lead to credit accumulation. Therefore, although the program may have affected other 
outcomes, we focus on earned college credits for the break-even analysis, as this outcome is 
of interest to practitioners and policymakers and more easily operationalized. 

Specifically, we focus on the additional number of college credits the program 
group would have had to earn for the program to break even in a cost-effectiveness 
framework. Although we only tracked students for two academic years, the logic of the 
break-even analysis can be extended beyond that time frame. In fact, students in the 
program group have an indefinite amount of time to earn the additional credits required for 
the program to break even. However, our discussion is limited to the two academic years 
following program participation. Although we conducted this analysis retroactively, it could 
be beneficial to conduct a break-even analysis prior to beginning a research study. After 
estimating the costs required to run a program, it is possible to estimate the magnitude of 
effect on any outcome of interest that would be required for the program to be considered 
cost effective. 

In general, there are three approaches that could be used to determine if the program 
was cost effective: cost-consequences analysis, cost-benefit analysis, and cost-effectiveness 
analysis. 5 We chose to use cost-effectiveness analysis (CEA), which determines whether it 
is cheaper to obtain educational outcomes using the developmental summer bridge 


4 CyFair and Kingwood, both part of the Lone Star College System, charged students $150 to participate 
in the program. This is the cost of a standard summer developmental course. Students could either pay the 
charge up front or have it deducted from their stipend. An overwhelming majority of students chose to have 
it deducted from their stipend, effectively lowering the stipend amount from $400 to $250. 

5 For a review of these three methods, see Levin and McEwan (2001). 
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programs or via standard educational offerings, also called the “business-as-usual” 
condition. The additional expense of a program may be justified if it yields more of the 
desired outcome (i.e., college credits earned). 6 Because we are focusing on only one 
outcome of the program — the additional number of college credits for the program group 
— its cost-effectiveness can be reported straightforwardly. 7 

Program Costs 

Costs for the program were calculated from the perspective of the college delivering 
the program. We considered this of greatest value to other institutions that may be looking 
to implement a developmental summer bridge program. All resources used by the colleges 
to implement the programs were counted, but resources required to develop the programs 
were not considered. 

Across the eight sites, the program cost was, on average, $1,319 per student (with a 
standard deviation of $502). There are two critical considerations in applying this cost 
measure to our analysis. First, this expenditure includes a stipend paid to students. We 
assume that this stipend is a cost — i.e., that the student would not have enrolled unless 
“paid” to do so. However, a portion of the stipend might be interpreted from a societal 
perspective as a transfer from the funder to the student. If the stipend is a transfer, it is not a 
cost, so this amount should be subtracted from the cost total. 

Second, the cost of the program should be net of the costs of business as usual. If 
some control group students are enrolled in other summer educational programs, the costs 
to society of these activities should be subtracted. However, it is hard to model the exact 
costs of other summer programs. Therefore, we assume that the full program resources 
made students choose the developmental summer bridge program rather than an alternative 
summer activity. This allows us to maintain the assumption that the cost per participating 
student is $1,319. We address these two assumptions — the nature of the stipend and the 
business-as-usual conditions — in our sensitivity analysis below. 


6 CEA also allows for comparison across individual developmental summer bridge programs to 
determine which yields the most positive outcomes per dollar spent; however, the sample sizes from each 
college are prohibitively small to determine this. 

7 

Typically, cost-effectiveness analysis is less useful because it requires cost-utility analysis in order 
to collapse all outcomes into a single index. In this case, there is only one outcome, so cost-effectiveness 
analysis may be used. 
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Cost of a College Credit 

For the cost-effectiveness analysis, we have to assign a monetary value to college 
credits. To determine this value, we calculate the cost of a college credit using the 
willingness-to-pay (WTP) method: If we had to “buy” a credit, what would we have to 
pay? In this case, the WTP value for credits is approximated by the amount currently 
spent on a credit in Texas. This includes not only what students pay in tuition and fees but 
also the public subsidies provided to colleges. In Texas, this amount is $338 (standard 
deviation $1 19). 8 

Cost-Effectiveness Analysis 

For the cost-effectiveness analysis, we determine how many additional college 
credits a program student would need to earn relative to a control student in order for the 
program to be “worth it,” i.e., effective in its ability to provide students with additional 
credit at the same cost as colleges typically spend per credit earned at the college. To 
calculate this, the average value of the stipend needs to be spread across the entire program 
group, not just the students who received the stipend. This lowers the average value of the 
stipend from $354 to $326. With this adjustment, the per-student cost of providing the 
program is $1,291. Based on the program costs and our WTP calculation, in order to be 
cost-effective (or break even), the program group would have to earn an additional 3.8 
college-level credits (3.8 = $ 1 ,29 1/$338) on average. 

Sensitivity Analysis: Alternative Assumptions 

In our sensitivity analysis, we test two key assumptions about costs: how to 
interpret the stipend and how to calculate the cost of the business-as-usual condition. It may 
be appropriate to interpret the stipend as a transfer from one group (the funders of 
developmental summer bridge program) to another group (the students). If so, then the 
stipend should not be counted as a cost of the program. The average value of the stipend 
across the entire program group was $326, so removing it reduces the cost of the program 
from $1,291 to $965 per program group member. 

In terms of defining the cost of the business-as-usual condition, we know that 16 
percent of the control group (Wathington et al., 2011, Table 4.1) enrolled in a standard 
developmental course during the summer. The costs of these courses are unknown. 


This is the average expenditure per credit across seven of the eight colleges, based on data from the 
Integrated Postsecondary Education Data System (IPEDS) on expenditures per full-time equivalent (FTE) 
(2008 data uprated to 201 1 dollars). Expenditures per FTE are adjusted to capture expenditures per credit 
attempted. One college does not have available IPEDS data. 
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Assuming that these courses are valued at three credits, the net cost of the program falls 
from $1,291 to $1,129, a reduction of $162. 

Combining these two assumptions (reducing the cost by a total of $488), a lower 
bound estimate of the per-student cost of the program is $803. If we use this value, which is 
slightly lower than the lowest cost program, the program group students needed to earn 2.4 
($803/$338) college-level credits more than the control group earned for the program to 
break even as measured by the ability to provide program students with additional credit at 
the typical cost to the college. 
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Chapter 6 


Conclusions and Implications 

This study sought to assess whether developmental summer bridge programs reduce 
students’ need for developmental course work upon fall matriculation and improve their 
outcomes in college. Recent research suggests that too few students advance from 
traditional developmental courses to college-level courses, which increases the pressure on 
colleges to identify and scale up successful developmental education initiatives. This study 
offers additional evidence on the effectiveness of one intervention designed to help 
developmental students succeed in college. This chapter summarizes the study’s key 
findings at the end of a two-year follow-up period discusses the implications and raises 
questions for further consideration. 


Review of Findings 

The following are the most important findings from the study: 

• The programs had no discernible effect on the average number of 
credits attempted or earned. Program group and control group students 
attempted the same number of credits (30.3). Students in the program 
group earned an average of 19.4 credits, and students in the control 
group earned an average of 19.9; the difference in their outcomes is not 
statistically significant. 

• The programs had an impact on first college-level course completion in 
math and writing that was evident in the year and a half following the 
program but no impact on first college-level course completion in 
reading during the same period. On average, students in the program 
group passed their first college-level math and writing courses at higher 
rates than students in the control group during this period. By the end 
of the two-year follow-up period, however, the differences between the 
two groups are no longer statistically significant. 

• There is no evidence that the programs impacted persistence. Over the 
course of the two-year follow-up period, students in the program group 
enrolled in an average of 3.3 semesters, and students in the control 
group enrolled in an average of 3.4 semesters; the difference in their 
outcomes is not statistically significant. 
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The Meaning of the Findings in Context 

The developmental summer bridge programs were effective in reducing the need for 
developmental education in college for the first year and a half following the program, as 
indicated by the greater number of program group students who passed their first college-level 
math and writing courses during the fall 2009, spring 2010, and fall 2010 semesters. This is a 
significant achievement and shows that the programs accomplished much of what they were 
intended to do. Yet the lack of detectable impacts on these same outcomes by the end of the 
spring 2011 semester suggests that the effects of the programs diminished in later years. 

The developmental bridge programs had no significant influence on two other 
important indicators of college student success — accumulation of credits and persistence. 
A careful review of the implementation research suggests that this finding is not due to poor 
or impartial program implementation by the colleges in this study. It is possible that the 
contrast between program and control group outcomes was reduced because 16 percent of 
the control group enrolled in summer courses during the summer of 2009, and these courses 
may have provided some of the same benefits as the developmental summer bridge 
program. However, this explanation probably does not fully explain the program’s lack of 
positive impacts on credit accumulation and persistence or its modest but transitory positive 
impacts on college-level course completion. 

It is also important to consider these findings in context. The students in this study 
were not selected to be a random sample of a larger population, and the impact estimates 
may not be generalizable to a larger population of colleges or students. While the students 
in our sample all started below college level in at least one subject, they were all highly 
motivated to take summer courses in preparation for college. Both program group and 
control group students enrolled in college at relatively high rates during the fall semester 
following the program. Since both groups of students were highly motivated, it may be that 
we should not expect to see large, long-term differences in their rates of college enrollment 
and persistence. 


Implications for Theory and Practice 

The findings in this report suggest that the developmental summer bridge program 
model contributed to positive outcomes in college-level course completion that were 
evident during the first year and a half following program completion. But it is important 
to note that these impacts faded over time. What is more, the programs do not appear to 
have encouraged persistence or credit completion. It may be the case that it was too 
ambitious to expect that the program model would produce significant impacts on these 
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outcomes. Perhaps the modest positive outcomes observed are as much as can be 
expected from a short, intensive summer program. 

First-year developmental education students may need additional transitional 
support beyond the developmental summer bridge program for greater impacts to be 
achieved. Unfortunately, we do not know exactly how much support developmental 
students require to advance to college-level work and continue to progress toward degree 
completion. A longer term intervention that continues to offer support to students after fall 
matriculation might increase the impact of developmental summer bridge programs; 
however, this implication is speculative at best. 

In addition, our research suggests that the completion of students’ first college-level 
courses in math or writing might not readily translate into the accumulation of more college 
credits. Thus, if the ultimate goal is college credential attainment, and credit accumulation 
is the indication of progress toward attaining a credential, then improving basic academic 
competencies through intensive remedial instruction and guidance about the college 
experience may not be the most effective way to promote attainment of this goal. 
Policymakers and practitioners may want to consider approaches that have a more direct 
link to credit accumulation and credential attainment. 

While college preparation is certainly associated with credential attainment, our 
study may imply that it is not possible to fully address students’ academic shortcomings 
during the relatively short timeframe of the summer bridge programs. Ideally, educators 
would be able to help students become more academically prepared while still in high 
school. Summer bridge programs could then attempt to remediate those educational deficits 
that remained, which could be fewer and milder. First-year experience programs might then 
assist students as they begin college, and ongoing supports in college-level courses could be 
offered. Alternatively, early interventions could span students’ time in both high school and 
college, or high schools and colleges could work in closer partnership to increase the 
number of students who graduate from high school college-ready . 1 

Finally, our break-even cost analysis suggests that students in developmental 
summer bridge programs would need to earn almost four additional college credits — or 
complete a little more than one additional college course (typically worth three credits) — 


1 For example, Virginia’s Capstone Courses in math and English, offered to students in their senior 
year of high school, were developed in partnership with several universities to better prepare students for 
college. The Getting Ready for Success program in Tacoma, Washington, seeks to boost low-income 
students’ college matriculation and success by offering students an array of services, including 
academics, mentoring, and monetary incentives. The program runs from the summer before students’ 
senior year of high school through their first year of postsecondary education. 
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for the program to provide students with credits at the typical cost per credit in Texas. The 
developmental bridge program model is relatively expensive and requires the availability of 
funds from outside of regular K-12 (average daily attendance) and higher education (FTE) 
public funding mechanisms. Educators may want to consider ways to reduce costs by 
embedding support programs such as these into the regular high school or college schedule. 


Implications for Research 

In addition to implications for theory and practice, there are additional 
considerations for future research. The results of this study suggest that, after the first year 
and a half, students were more likely to have passed introductory math and writing. Other 
quasi-experimental summer bridge studies demonstrate positive findings (Santa Rita & 
Bacote, 1997; Strayhorn, 2011), but program students are seldom followed for as long as 
two years as they were in the current study. It is possible that an even longer follow-up is 
required to fully assess the effectiveness of these programs, which may ultimately turn out 
to influence longer term outcomes such as credential completion or career success. 

Further, because the program features were tested together as a bundle, it is 
impossible to know if any of the four program components contributed more than others to 
the program’s impacts on student outcomes. Research could be conducted that permits a 
more fine-grained analysis of the separate features of the program. Similarly, we might 
collect more information on whether these programs are likely to be more effective in math, 
English, or another subject area. 

Researchers may also wish to consider whether other strategies could be used in 
college to improve students’ skills in math and English. Would these strategies complement 
traditional developmental education or summer bridges, or would they replace them? 
Additional research could be undertaken to cast light on the specific design features and 
strategies that could influence outcomes, whether implemented separately or as a pieces of a 
coordinated effort. 

Finally, more information is needed on whether there are groups of students who 
are more likely to benefit from a developmental summer bridge program. For example, are 
there levels of developmental need that can be addressed effectively by a developmental 
summer bridge program? Are these relatively short programs best suited for students who 
are almost college ready or for those with more serious deficits? Further research would be 
required to answer these questions. 
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Concluding Thoughts 

The developmental summer bridge programs studied here were found to have 
modest positive impacts in the short-term. What is clear from this study and other 
developmental education research is that simple, short-term interventions yielding strong, 
long-term effects are difficult to find. We offer two suggestions for action in advancing the 
work of supporting underprepared students: (1) introducing new partnerships between high 
schools and colleges that reduce the need for remediation in college and (2) providing more 
support and transitional experiences to help students reach and sustain attainment goals. 
Because educational attainment is the result of a long process influenced by many factors, 
providing supports to students that span their years in high school and college may help 
them to develop the skills and knowledge required for postsecondary success. 
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Appendix A 

Supplementary Tables 




Texas Developmental Summer Bridge Programs 
Appendix Table A.l 

Progression Through Developmental Education Sequences in Math, Reading, and Writing 

Two Years After Random Assignment 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Attempted any course (%) 






Math 

84.3 

83.2 

1.1 

.61 

2.0 

Reading 

86.3 

85.7 

0.6 

.77 

1.9 

Writing 

86.4 

83.4 

3.0 

.13 

2.0 

Sample size (total = 1,318) 

793 

525 





Math 




Progression index 

0.8 

1.0 

-0.1 



Sample size (total = 1,105) 

668 

437 





Reading 




Progression index 

0.2 

0.3 

0.0 



Sample size (total = 1,134) 

684 

450 





Writing 




Progression index 

0.2 

0.2 

0.0 



Sample size (total = 1,123) 

685 

438 





SOURCE: NCPR calculations from Texas Higher Education Coordinating Board data and college transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. A two-tailed f-test was applied to 
differences between research groups. Estimates are adjusted by site. 

*p < .10. **p < .05. ***p < .01. 
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Texas Developmental Summer Bridge Programs 
Appendix Table A.2 


Cumulative Percentage of Students Passing College-Level Courses 
in Math, Reading, and Writing by Semester 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Passed first college-level math course (%) 

Fall 2009 

10.7 

4.8 

5 9*** 

.00 

1.5 

Spring 2010 

32.3 

22.8 

9 4 *** 

.00 

2.4 

Summer 2010 

35.6 

28.2 

7 4*** 

.00 

2.5 

Fall 2010 

42.6 

36.9 

5.7** 

.03 

2.6 

Spring 2011 

46.5 

43.0 

3.5 

.19 

2.7 

Passed first college-level reading course (%) 

Fall 2009 

32.1 

28.8 

3.3 

.17 

2.4 

Spring 2010 

62.8 

57.9 

4.9* 

.06 

2.6 

Summer 2010 

65.1 

61.3 

3.7 

.14 

2.5 

Fall 2010 

70.1 

66.5 

3.6 

.14 

2.5 

Spring 2011 

72.6 

71.6 

1.0 

.68 

2.4 

Passed first college-level writing course (%) 

Fall 2009 

32.9 

28.8 

4.1* 

.10 

2.5 

Spring 2010 

63.2 

58.2 

5.0* 

.06 

2.6 

Summer 2010 

64.8 

59.6 

5.3** 

.04 

2.6 

Fall 2010 

69.0 

64.5 

4.5* 

.08 

2.5 

Spring 2011 

71.7 

68.3 

3.3 

.18 

2.5 

Sample size (total = 1,318) 

793 

525 





SOURCE: NCPR calculations from Texas Higher Education Coordinating Board data and college transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. A two-tailed f-test was applied to 
differences between research groups. Estimates are adjusted by site. 

*p < .10. **p < .05. ***p < .01. 
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Texas Developmental Summer Bridge Programs 
Appendix Table A.3 

Financial Aid Outcomes in the 2010-11 Academic Year 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Sprint; 2010 

Registered at summer bridge school (%) 

66.7 

64.6 

2.1 

.41 

2.6 

Received financial aid (%) 

58.4 

55.0 

3.4 

.22 

2.7 

Pell grant 

46.8 

45.2 

1.6 

.57 

2.7 

Other aid 

50.5 

49.5 

1.0 

.73 

2.8 

Average aid received ($) 

1,014 

1,002 

12.5 

.84 

62.5 

Pell grant 

918 

897 

21.0 

.74 

62.3 

Other aid 

404 

416 

-11.3 

.73 

32.8 

Fall 2010 

Registered at summer bridge school (%) 

54.1 

51.4 

2.6 

.33 

2.7 

Received financial aid (%) 

33.4 

32.5 

0.9 

.73 

2.6 

Pell grant 

26.7 

25.1 

1.6 

.49 

2.3 

Other aid 

24.1 

21.7 

2.4 

.32 

2.4 

Average aid received ($) 

699 

692 

7.0 

.92 

65.7 

Pell grant 

614 

610 

4.2 

.95 

63.7 

Other aid 

219 

210 

9.4 

.77 

32.3 

Soring 2011 

Registered at summer bridge school (%) 

48.4 

50.5 

-2.1 

.44 

2.7 

Received financial aid (%) 

34.0 

34.5 

-0.5 

.84 

2.6 

Pell grant 

25.5 

25.6 

-0.1 

.96 

2.3 

Other aid 

23.5 

22.7 

0.8 

.74 

2.4 

Average aid received ($) 

749 

760 

-11.7 

.87 

69.6 

Pell grant 

602 

637 

-34.5 

.59 

64.7 

Other aid 

271 

251 

20.5 

.59 

37.6 

Sample size (total = 1.318) 

793 

525 





SOURCE: NCPR calculations from college transcript and financial aid data. 


NOTES: Rounding may cause slight discrepancies in sums and differences. Aid amounts are rounded to the nearest 
dollar. A two-tailed ?-test was applied to differences between research groups. Estimates are adjusted by site. 

*p < .10. **p < .05. ***p < .01. 
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Appendix Table A.4 

Financial Aid Outcomes at Alamo Colleges and Non- Alamo Colleges in the 2009-10 Academic Year 


Non-Alamo Colleges Alamo Colleges 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Difference 

Between 

Subgroups 

Fall 2009 

Registered at summer bridge school (%) 

74.7 

72.5 

2.2 

.47 

3.1 

66.3 

66.2 

0.2 

.97 

4.4 


Received financial aid (%) 

57.4 

56.0 

1.4 

.70 

3.5 

62.2 

50.8 

11.5** 

.01 

4.5 

t 

Pell grant 

42.0 

42.5 

-0.4 

.90 

3.4 

55.4 

45.7 

9.8** 

.03 

4.6 

t 

Other aid 

51.3 

51.4 

-0.1 

.98 

3.5 

46.9 

38.5 

8.4* 

.07 

4.6 


Average aid received ($) 

1,068 

1,134 

-66.2 

.43 

82.9 

1,029 

795 

234.3** 

.02 

99.1 

tt 

Pell grant 

930 

975 

-44.5 

.59 

82.5 

996 

772 

223.4** 

.03 

99.8 

tt 

Other aid 

418 

487 

-68.9 

.14 

46.7 

249 

174 

75.3** 

.03 

34.2 

tt 

Soring 2010 

Registered at summer bridge school (%) 

72.5 

70.3 

2.2 

.48 

3.1 

56.8 

54.8 

2.0 

.66 

4.6 


Received financial aid (%) 

55.7 

56.3 

-0.6 

.87 

3.5 

62.9 

52.8 

10.1** 

.03 

4.5 

t 

Pell grant 

41.2 

43.4 

-2.2 

.53 

3.4 

56.1 

48.2 

7.9* 

.09 

4.6 

t 

Other aid 

50.5 

52.4 

-1.8 

.60 

3.5 

50.3 

44.6 

5.7 

.22 

4.6 


Average aid received ($) 

1,025 

1,124 

-98.8 

.23 

81.9 

996 

795 

201.0** 

.04 

95.1 

tt 

Pell grant 

890 

970 

-79.8 

.33 

81.1 

964 

773 

191.7** 

.05 

96.3 

tt 

Other aid 

443 

515 

-72.1 

.13 

47.4 

340 

248 

91.7** 

.01 

36.4 

ttt 
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Financial Aid Outcomes at Alamo Colleges and Non- Alamo Colleges in the 2009-10 Academic Year 


Non-Alamo Colleges Alamo Colleges 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Difference 

Between 

Subgroups 

Fall 2010 

Registered at summer bridge school (%) 

58.9 

57.0 

1.9 

.57 

3.4 

45.9 

42.0 

3.9 

.40 

4.6 


Received financial aid (%) 

33.8 

33.4 

0.4 

.91 

3.3 

32.6 

30.8 

1.8 

.66 

4.2 


Pell grant 

23.9 

23.8 

0.2 

.95 

2.8 

31.3 

27.2 

4.1 

.33 

4.1 


Other aid 

23.4 

21.8 

1.6 

.59 

3.0 

25.2 

21.6 

3.6 

.35 

3.9 


Average aid received ($) 

757 

785 

-27.9 

.76 

90.7 

601 

535 

66.1 

.45 

88.1 


Pell grant 

633 

667 

-34.8 

.69 

87.0 

583 

513 

70.2 

.43 

88.2 


Other aid 

256 

258 

-1.4 

.98 

48.3 

157 

129 

27.5 

.36 

30.2 


Serine 2011 

Registered at summer bridge school (%) 

53.5 

56.1 

-2.6 

.44 

3.4 

39.8 

41.0 

-1.2 

.79 

4.5 


Received financial aid (%) 

33.6 

32.8 

0.8 

.82 

3.3 

34.7 

37.4 

-2.8 

.54 

4.4 


Pell grant 

24.1 

23.5 

0.6 

.82 

2.8 

27.9 

29.3 

-1.4 

.73 

4.1 


Other aid 

22.4 

21.5 

0.9 

.76 

2.9 

25.2 

24.6 

0.6 

.88 

4.0 


Average aid received ($) 

762 

782 

-19.4 

.83 

90.3 

726 

725 

1.2 

.99 

108.5 


Pell grant 

638 

664 

-26.0 

.76 

86.7 

542 

591 

-49.0 

.60 

93.8 


Other aid 

252 

259 

-6.6 

.89 

47.8 

303 

237 

66.4 

.27 

60.7 


Sample size (total = 1,318) 

499 

330 




294 

195 






SOURCE: NCPR calculations from college transcript and financial aid data. 

NOTES: The Alamo Community College System consists of Palo Alto College, San Antonio College, and St. Phillip's Academy. Rounding may cause slight discrepancies in 
sums and differences. Aid amounts are rounded to the nearest dollar. A two-tailed t-test was applied to differences between research groups. Estimates are adjusted by site. 

*p < .10. **p < .05. ***p < .01. 

Significant differences between subgroups denoted as follows: ' 1 p < .01; 1 f p < .05; 'p < .10. 
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Key Academic Outcomes by Gender Two Years After Random Assignment 


Outcome 



Males 





Females 




Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Difference 

Between 

Subgroups 

Semesters registered at any college 11 

3.3 

3.5 

-0.2 

.18 

0.1 

3.3 

3.3 

0.0 

.96 

0.1 


Total credits attempted 11 

31.1 

31.5 

-0.3 

.86 

1.9 

29.9 

29.4 

0.5 

.74 

1.5 


College-level 

25.1 

25.2 

0.0 

.98 

1.8 

23.7 

22.4 

1.3 

.35 

1.4 


Developmental 

6.0 

6.3 

-0.3 

.63 

0.6 

6.2 

7.0 

-0.8 

.11 

0.5 


Total credits earned 

19.9 

20.4 

-0.5 

.74 

1.6 

19.3 

19.4 

-0.1 

.93 

1.3 


College-level 

16.6 

16.7 

-0.1 

.92 

1.5 

15.7 

15.2 

0.5 

.67 

1.2 


Developmental 

3.3 

3.7 

-0.4 

.35 

0.4 

3.6 

4.2 

-0.6* 

.08 

0.3 


Passed first college-level math course (%) 

45.9 

42.6 

3.3 

.46 

4.5 

47.2 

43.4 

3.7 

.28 

3.4 


Passed first college-level reading course (%) 

73.2 

70.1 

3.1 

.43 

3.9 

72.6 

73.2 

-0.5 

.86 

3.1 


Passed first college-level writing course (%) 

70.9 

67.5 

3.4 

.41 

4.1 

72.5 

69.3 

3.2 

.32 

3.2 


Sample size (total = 1,299) 

293 

196 




489 

321 






SOURCE: NCPR calculations from Texas Higher Education Coordinating Board data and college transcript data, unless otherwise noted. 


NOTES: Rounding may cause slight discrepancies in sums and differences. A two-tailed t-test was applied to differences between research groups. Estimates are adjusted by site. 
*p < .10. **p < .05. ***p < .01. 

Significant differences between subgroups denoted as follows: < .01; 1 ’p < .05; 'p < .10. 

“The number of semesters registered at any college is based on NCPR calculations from Texas Higher Education Coordinating Board data, college transcript data. National 
Student Clearinghouse data, and student survey data. 

b Credits attempted and earned are based on NCPR calculations from college transcript data and include summer 2009 and summer 2010 terms. 
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Key Academic Outcomes by Mother’s Educational Attainment Two Years After Random Assignment 


Lft 

VO 


No College At Least Some College 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Dinerence 

Between 

Subgroups 

Semesters registered at any college 11 

3.2 

3.4 

-0.2 

.11 

0.1 

3.6 

3.6 

0.0 

.93 

0.1 


Total credits attempted 13 

29.1 

29.6 

-0.5 

.76 

1.6 

32.5 

33.1 

-0.6 

.76 

2.0 


College-level 

22.8 

22.7 

0.2 

.91 

1.5 

26.8 

26.6 

0.2 

.91 

1.8 


Developmental 

6.2 

6.9 

-0.7 

.20 

0.5 

5.7 

6.5 

-0.8 

.17 

0.6 


Total credits earned 

17.9 

18.7 

-0.8 

.56 

1.3 

21.4 

23.1 

-1.7 

.32 

1.7 


College-level 

14.4 

14.5 

0.0 

.98 

1.2 

18.3 

19.2 

-1.0 

.55 

1.6 


Developmental 

3.5 

4.2 

-0 7** 

.05 

0.4 

3.1 

3.9 

-0.7* 

.08 

0.4 


Passed first college-level math course (%) 

43.3 

37.1 

6.2* 

.09 

3.6 

53.2 

51.2 

2.0 

.67 

4.7 


Passed first college-level reading course (%) 

67.7 

68.4 

-0.7 

.83 

3.4 

81.6 

79.5 

2.1 

.57 

3.7 


Passed first college-level writing course (%) 

69.0 

65.3 

3.7 

.28 

3.5 

76.4 

73.8 

2.6 

.52 

4.1 


Sample size (total = 1,181) 

464 

291 




251 

175 






SOURCE: NCPR calculations from Texas Higher Education Coordinating Board data and college transcript data, unless otherwise noted. 


NOTES: Rounding may cause slight discrepancies in sums and differences. A two-tailed t-test was applied to differences between research groups. Estimates are adjusted by site. 
*p < .10. **p < .05. ***p < .01. 

Significant differences between subgroups denoted as follows: ' 1 'p < .01; 1 l p < .05; 'p < .10. 

‘‘The number of semesters registered at any college is based on NCPR calculations from Texas Higher Education Coordinating Board data, college transcript data. National 
Student Clearinghouse data, and student survey data. 

b Credits attempted and earned are based on NCPR calculations from college transcript data and include summer 2009 and summer 2010 terms. 
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Key Academic Outcomes by Program Type Two Years After Random Assignment 


Outcome 



Freestanding 





Course Based 



Difference 

Between 

Subgroups 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P 

Standard 

Error 

Semesters registered at any college 11 

3.3 

3.4 

-0.1 

.25 

0.1 

3.3 

3.3 

0.0 

.94 

0.2 


Total credits attempted 11 

29.6 

30.5 

-0.9 

.52 

1.4 

31.7 

29.7 

1.9 

.36 

2.1 


College-level 

23.9 

23.7 

0.2 

.87 

1.3 

24.7 

23.2 

1.5 

.42 

1.9 


Developmental 

5.7 

6.8 

-1.2** 

.01 

0.5 

7.0 

6.6 

0.4 

.55 

0.7 

t 

Total credits earned 

18.7 

20.4 

-1.7 

.17 

1.2 

20.7 

19.0 

1.8 

.30 

1.7 

t 

College-level 

15.5 

16.2 

-0.7 

.54 

1.1 

16.8 

15.3 

1.5 

.34 

1.6 


Developmental 

3.2 

4.2 


.00 

0.3 

4.0 

3.7 

0.3 

.53 

0.5 

tt 

Passed first college-level math course (%) 

44.0 

40.9 

3.1 

.36 

3.3 

51.5 

47.1 

4.4 

.33 

4.5 


Passed first college-level reading course (%) 

72.5 

71.1 

1.4 

.65 

3.0 

73.0 

72.7 

0.3 

.94 

4.1 


Passed first college-level writing course (%) 

70.1 

67.8 

2.2 

.48 

3.1 

74.8 

69.3 

5.6 

.18 

4.2 


Sample size (total = 1,181) 

527 

349 




266 

176 






SOURCE: NCPR calculations from Texas Higher Education Coordinating Board data and college transcript data, unless otherwise noted. 


NOTES: Rounding may cause slight discrepancies in sums and differences. A two-tailed t-test was applied to differences between research groups. Estimates are adjusted by site. 
*p < .10. **p < .05. ***p < .01. 

Significant differences between subgroups denoted as follows: ' 1 'p < .01; 1 l p < .05; 'p < .10. 

‘‘The number of semesters registered at any college is based on NCPR calculations from Texas Higher Education Coordinating Board data, college transcript data. National 
Student Clearinghouse data, and student survey data. 

b Credits attempted and earned are based on NCPR calculations from college transcript data and include summer 2009 and summer 2010 terms. 



Appendix B 

Individual Program Descriptions 




The Developmental Summer Bridge Colleges 

The eight institutions selected for inclusion in the study are El Paso Community 
College, Lone Star College-CyFair, Lone Star College-Kingwood, South Texas College, 
Texas A&M International University, Palo Alto College, San Antonio College, and St. 
Philip’s College. Of their developmental summer bridge programs, four were course- 
based, while another four were freestanding. Course-based programs were based on 
standard developmental courses, modified or condensed to create a shorter, more 
intensive experience. Students enrolled in course-based programs also received additional 
academic supports, guidance in college knowledge, and a $400 stipend to supplement 
their summer course work. Students were typically awarded developmental education 
credit 1 (below the college level) for completion of these courses. Freestanding programs 
were designed to provide students the opportunity to advance multiple skill levels by 
offering basic skills instruction and were not based on a specific course. These programs 
did not require enrollment in a summer course and did not award any form of credit. A 
brief description of each college and its developmental summer bridge program is 
provided in the pages that follow. 

El Paso Community College 

El Paso Community College (EPCC) is an urban institution that was established in 
1969. El Paso County Community College District includes five campuses; its 
developmental summer bridge program, Project Dream, was operated at four of the 
campuses. In fall 2008, EPCC had a full-time-equivalent enrollment of 15,201 students. 
The student population was about 85 percent Hispanic, 8 percent White non-Hispanic, 
and 2 percent African American. These percentages closely reflect the demographics of 
El Paso County. 

Project Dream has operated since 2006 and runs for five weeks each summer. The 
program was one of three in the study to receive funding through a grant from the 
THECB for 2009-11. Project Dream, a freestanding program, enrolled students in 
coursework in math and reading, as well as a Dream Class that incorporated writing, 
cultural awareness, and college knowledge. Program instructors, trained in pedagogy, 
worked to offer contextualized instruction. Program students were mentored by older 
college students. 


developmental education credits are awarded for financial aid eligibility purposes but cannot be 
applied toward a degree. 
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Lone Star College-CyFair 

Lone Star College-CyFair, established in 2003, is the newest of five campuses of 
the Lone Star College System (LSCS), which serves the suburbs of Houston. In fall 2008, 
Lone Star College-CyFair had a full-time-equivalent enrollment of 6,709 students. The 
student population in 2008 was about 40 percent White non-Hispanic, 30 percent Hispanic, 
10 percent African American, and 10 percent Asian American. 

The Summer Bridge Program has operated since 2006 and runs for four weeks. In 
this course-based program, students participated in one developmental education course in 
math, reading, or writing. A11 had placed into the highest-level developmental course in the 
subject area. Other important elements of the program included the presence of tutors in 
each class, the use of MyMathLab and MyWritingLab software, college knowledge 
workshops, and individually assigned mentors for every student. 

Lone Star College-Kingwood 

Lone Star College-Kingwood, which is also part of the five-campus LSCS, was 
established in 1984. In fall 2008, Lone Star College-Kingwood had a full-time-equivalent 
enrollment of 3,820 students. The student population at Lone Star College-Kingwood in 
2008 was about 55 percent White non-Hispanic, 20 percent Hispanic, and 12 percent 
African American. 

The college has offered the four-week Summer Bridge Program since 2006. The 
course -based program offered classes in math and writing, and students took only one 
subject. Students in the writing option had class for four hours per day, while those in math 
had class for three hours per day. Both options were for students whose placement test 
scores put them at the highest level of the developmental course sequence. Important 
elements of the program included tutors who were present in each class and available to 
provide students with extra help before and after class, four one-hour workshops on study 
skills and college knowledge, and College Connection mentors with structured time to 
interact with students. 

South Texas College 

South Texas College, founded in 1993, recently was designated a four-year college 
as it began to offer baccalaureate degrees. It is located in McAllen, Texas, near the border 
of Mexico. In fall 2008, South Texas had a full-time-equivalent enrollment of 12,644 
students. The student population at South Texas is primarily Hispanic (about 95 percent). 
Interestingly, the Hispanic student population of South Texas College is higher than the 
proportion of Hispanics in the city of McAllen. 
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The summer bridge program, called Jumpstart, was established in 2007 (the 
program did not run in 2008). A freestanding program, Jumpstart runs for four weeks and 
focuses only on developmental math instruction. Significant elements of the program 
included weekly college knowledge workshops, use of MyMathLab to supplement 
instruction, tutors in the classrooms and labs, and daily journal writing to informally 
develop writing skills. Students in the math courses were grouped without regard for the 
level of remediation needed. 

Texas A&M International University 

Texas A&M International University (TAMIU) is a four-year institution that first 
started to accept students in 1970 and became a campus of the Texas A&M University 
System in 1989. It is the only university in the NCPR study and one of three institutions in 
the study to receive funding through a grant from the THECB. Located in Laredo on the 
border of Mexico, TAMIU has a heavily Hispanic student population (over 90 percent). In 
fall 2008, TAMIU had a full-time-equivalent enrollment of 4,222 students. 

The freestanding summer bridge program at TAMIU started in 2007. In previous 
summers, the program offered English and math, but for the summer 2009 program, the 
staff decided to focus solely on developmental math instruction with all levels mixed 
together in each section. The program was called Intensive College Math Prep. In addition 
to the intensive, accelerated math instruction, other important elements of the program were 
tutor-administered math lab hours, weekly meetings with older-student mentors, and weekly 
college knowledge presentations. The program ran for five weeks. 

Palo Alto College 

Palo Alto College, a member of the Alamo Community College District (ACCD) in 
San Antonio, was established in 1983. In fall 2008, the college had a full-time-equivalent 
enrollment of 4,486 students. The student population in 2008 was about 66 percent 
Hispanic, 30 percent White non-Hispanic, and 2 percent African American. The 
demographics of Palo Alto College closely resemble those of greater San Antonio, where 
59 percent of the population are Hispanic or Latino of any race. 

In the past, Palo Alto College ran summer bridge programs that focused on 
SAT/ACT preparation. In the summer of 2009, Palo Alto College started the Early Start 
program, which offered only developmental math in a course-based format. Students 
attended class for four weeks. Distinctive features of the program included tutors in each 
class, provision of breakfast and lunch to all students, a cohort model, and a student 
development course. 
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San Antonio College 

San Antonio College, also a member of ACCD, was established in 1925 and in fall 
2008 had a full-time-equivalent enrollment of 12,226 students. The student population is 
about 48 percent Hispanic, 42 percent White non-Hispanic, and 5 percent African 
American. The summer bridge program at San Antonio College, established in 1999, is 
called Senior Summer. 

Senior Summer allows students to enroll in developmental math, reading, and 
writing as well as college-level courses. Participating students chose two courses from the 
regular college summer offerings. The program ran three and a half hours per day for five 
weeks, with an additional hour two days each week. The program staff emphasized that 
they recruit students who are often overlooked as “college material.” Important elements of 
Senior Summer included enrollment in actual developmental-level courses, a pared-down 
version of a student development course, assistance with financial aid forms, and fostering 
college-going attitudes among students. 

San Antonio College had difficulty recruiting students for the 2009 summer 
program. In previous years, program staff had worked closely with individual students 
during the recruitment process to provide assistance in completing financial aid forms, 
which is a key part of this course-based program because students pay tuition. Because of 
the random assignment design, it became more complicated to work effectively with 
individual students. 

St. Philip’s College 

St. Philip’s College, another ACCD member in San Antonio, is a historically Black 
college and Hispanic-serving institution that was founded in 1898 and became a public two- 
year institution in 1942. In fall 2008, St. Philip’s College had a full-time-equivalent 
enrollment of 5,533 students. The student population is about 47 percent Hispanic, 34 
percent White non-Hispanic, and 16 percent African American. The freestanding summer 
bridge program is called the Fresh X Summer Program. 

The Fresh X program provides developmental education instruction in math, 
reading, and writing. Students took courses in one, two, or all three developmental subject 
areas based on remediation needs as indicated by ACCUPLACER 2 test scores. The contact 
hours of the program varied depending on the number of courses taken. The program ran 
five days per week for four weeks. Fresh X was marketed as an ACCUPLACER refresher 
course, and the goal of the program was to improve students’ scores on the test. The 


2 

ACCUPLACER is an adaptive placement test developed by the College Board. 
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instruction was heavily focused on passing the test as a result. Other important elements of 
the Fresh X program included support from the math, writing, and reading labs; a credit- 
bearing student development course; and use of adaptive learning software to supplement 
math instruction. 


Program Updates: Summer 2010 

In 2010, five of the eight colleges ran developmental summer bridge programs 
similar to those they offered in 2009. At each of these sites, program size was considerably 
smaller, in large part because the colleges lacked NCPR funding and because institutional 
budget cuts constrained program capacity. Recruitment also presented challenges for the 
colleges, since none of the programs was able to offer the $400 stipend provided in 2009. 
Two colleges offered a stipend of up to $200, one college offered tuition reimbursement, 
and another college provided books and calculators for regular attendance, but the financial 
incentive to students was in no way comparable to that of 2009. 

Colleges did learn a variety of recruitment techniques by way of their participation 
in the 2009 study. In particular, administrators remarked that direct contact with the high 
schools is integral to any college recruitment effort. As a result, College Connection 
advisors played a key role in recruiting 2010 summer program participants. One program 
administrator believed that the reputational success of the 2009 program aided in recruiting 
participants for the 2010 program. 

While most of the colleges did not change their instructional offerings in significant 
ways, there were some notable changes. One college added a developmental reading course 
to its program, while another institution revised all reading and writing curricula within its 
freestanding program. A few colleges changed their math software programs or provided 
more training for mentors and tutors. One college also provided faculty development in 
2010 for the first time. 

The three colleges that did not offer programs in 2010 cited diminished resources as 
the primary reason for eliminating them. Each of the colleges decided to devote their 
limited resources and primary attention to strengthening other college readiness or 
developmental student success programs (e.g., career academies, success courses for 
students in developmental math, success courses for students on academic probation, etc.). 
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